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(57) Abstract: An opinion research system is adapted to collect, analyze and report real time opinion data regarding a variety of 
media events. A media management server streams the live or on-demand events to a panel of participants over a computer network 
such as, the global Internet or an organization's intranet. Each participant receives the media streams on a client apparatus (e.g., 
personal computer, personal digital assistant (PDA), telephone, television and the like) and interacts with an input device for a dis- 
play GUI to indicate an opinion about the media streams. The media streams can be sent to the same apparatus used to receive the 
opinion data. Alternatively, the media streams can be sent to a separate apparatus independent of the polling client apparatus. A 
polling API interfaces between the input device and polling client apparatus to provide real time visual feedback to the participant. 
Accordingly, the participant's current opinion can be illustrated on a dial control for the display GUI. Periodic scheduled communi- 
cations of the current opinion arc routed to a collection and display server which generates a composite opinion value for the entire 
panel. The collection and display server can also generate demographically filtered views of the composite opinion from participant 
demographics stored in a user demographic database. The opinion data also includes lime stamps that synchronizes each opinion 
value with the media stream. As such, statistical analysis can be performed on individual media frames, i.e. topics, scenes, issues, 
and the like, from the media event. The composite results, filtered composite results and graphical representations of the results arc 
routed to the participant's display GUIs. 
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System, Method and Computer Program Product for 
Collection of Opinion Data 



Background of the Invention 
L Field of the Invention 

The present invention relates generally to opinion researching, and more 
particularly to a technique for polling and presenting opinion data over a 
computer network. 

2. Related Art 

Opinion gathering tools, such as dial polls, are among the mainstays of the 
opinion research industry. Current methodologies, technology and products limit 
the use of these powerful tools to restricted environments and relatively small 
groups or panel sizes. 

A dial poll is an opinion research tool for capturing reactions to events as 
they happen. In a typical dial poll, a panel of respondents or participants is 
selected and their demographics are captured. The participants are equipped with 
an instrument in the form of a dial that they hold in their hands as they watch an 
event or presentation. The panel members are instructed to rotate the dial to 
indicate their approval or disapproval of the presentation as it unfolds. 

The dials are typically wireless devices communicating to a data 
collection device interfaced to a personal computer. The real time opinions are 
captured from the panel for analysis and are, in many cases, displayed as overlaid 
graphics over the presentation or event video. 

Underlying databases of user demographic information provide for 
selected data views of the audience response. Extensive post event analysis is 
carried out by studying behavioral data associated with key elements of the 
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presentation. Extensive demographic decomposition as well as specific segment 
analysis is routinely carried out. 

Currently an estimated four billion dollars are spent annually on opinion 
research. Of this figure, dial poll research accounts for less than 100 million 
dollars. The reasons for this is not in the benefits gleaned in the unique opinion 
research data gathered, but in the difficulties and expenses associated with real 
time opinion research. 

Traditional dial poll systems require a panel of 30 to 50 participants to be 
recruited and brought to a central location where the specialized equipment is 
located. Recruiting the panel entails logistics, a staff and typically some form of 
compensation to each panel participant. As such, a typical dial poll, using a panel 
of thirty to fifty participants, can cost between fifteen to thirty thousand dollars 
in production costs, require specially skilled personnel, a suitable facility and 
specialized dedicated equipment. 

Although larger panel sizes can be used, they tend to represent production 
and budgetary restraints. In other words, substantial planning, management and 
staff are required to facilitate a dial poll. 

What is needed is a system and method for performing real time opinion 
research that overcomes the cost and complexity issues associated with traditional 
dial poll methodologies. 

Summary of the Invention 

The present invention is directed to an opinion research system and 
method for collecting, analyzing and reporting real time opinion data regarding 
a variety of media events. Media events can include speeches, debates, 
presentations, contests, pageants, newscasts, sporting events, motion picture 
screenings, commercial testings, and the like. 
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A media management server streams the live or on-demand events to a 
panel of respondents or participants over a computer network such as, the global 
Internet or an organization's intranet. To route the media streams, the server can 
use satellite, terrestrial, radio, microwave and any other form or method of 
transmission. 

Each participant receives the media streams on a client apparatus and 
interacts with an input device for a display (e.g., graphical user interface "GUI") 
to indicate an opinion about the media streams. In one embodiment of the present 
invention, the media streams are sent to the same client apparatus used to receive 
the opinion data. Hence, the media streaming and polling client apparatuses are 
the same device. In another embodiment, the media streams are sent to a separate 
apparatus that is independent of the polling client apparatus. 

A polling application program interface (API) interacts between the input 
device and polling client apparatus to provide real time visual feedback to the 
15 participant. Accordingly, the participant's current opinion can be illustrated on a 

dial control for the GUI. 

Periodic scheduled communications of the current opinion are routed to 
a collection and display server that generates a composite opinion value for the 
entire panel. The collection and display server can also generate demographically 
filtered views of the composite opinion from participant demographics stored in 
a user demographic database. The opinion data also includes time stamps that 
synchronizes each opinion value with the media stream. As such, statistical 
analysis can be performed on individual media frames, i.e., topics, scenes, issues, 
and the like, from the media event The composite results, filtered composite 
results and graphical representations of the results (collectively referred to herein 
as composite opinion data) are routed to the participants' display GUIs. 

It is, therefore, a feature of the present invention that it provides real time 
opinion research for media events over a computer network. 



20 
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It is a further feature of the present invention that the panel's response 
data (e.g., dial data or other indicator) is synchronized with the video, audio or 
multimedia feeds of the event with minimum lag. 

It is a further feature of the present invention that it supports the polling 
5 of live, time delayed and on-demand media events. 

It is a further feature of the present invention that demographically filtered 
views of the opinion data can be generated in real time from a user demographic 
database. 

It is yet a further feature of the present invention that multiple and 
10 simultaneous demographically driven views of the synchronized data and video 

data can be generated in real time or during delayed analysis. 

An advantage of the present invention is a substantial reduction in the 
costs of performing traditional dial polls. By collecting opinion data over a 
computer network, the poll panel is no longer required to be brought to a 
15 centralized location. Costs associated with procuring specialized equipment, 

facilities and staff are also mitigated. 

A further advantage of the present invention is that the panel size can be 
scaled to hundreds of thousands of simultaneous participants, without the added 
costs of procuring larger facilities and equipment and additional staff. 
20 A still further advantage of the present invention is the ability to capture 

accurate and synchronized panel responses and provide instant access and 
analysis by using real time demographic views. 

Brief Description of the Figures 

The accompanying drawings, which are incorporated herein and form part 
25 of the specification, illustrate the present invention and, together with the 

description, further serve to explain the principles of the invention and to enable 
a person skilled in the pertinent art to make and use the invention. In the 
drawings, like reference numbers indicate identical or functionally similar 
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elements. Additionally, the left-most digit(s) of a reference number identifies the 
drawing in which the reference number first appears. 

FIG. 1 illustrates a high level block diagram of the configuration of a real 
time polling system according to one embodiment of the present invention. 

FIG. 2 illustrates a graphical user interface (GUI) for a client apparatus 
according to an embodiment of the present invention according to one 
embodiment of the present invention. 

FIG. 2a illustrates an analog-based graphical representation of a polling 
indicator according to one embodiment of the present invention. 

FIG. 2b illustrates a discrete value-based graphical representation of a 
polling indicator according to one embodiment of the present invention. 

FIG. 3 illustrates a high level block diagram of the configuration of a 
mediamanagement server according to one embodiment of the present invention. 

FIG. 4 illustrates a high level block diagram of the configuration of a load 
balancer according to one embodiment of the present invention. 

FIG. 5 illustrates a high level block diagram of the configuration of a 
polling server according to one embodiment of the present invention. 

FIG. 6 illustrates a master data table resident in the memory of a polling 
server according to one embodiment of the present invention. 

FIG. 7 illustrates a high level block diagram of the configuration of a 
collection and display server according to one embodiment of the present 
invention. 

FIG. 8 is a block diagram of an example computer system useful for 
implementing the present invention. 

FIG. 9 illustrates a high level operational flow diagram for the steps 
involved in collecting opinion data according to one embodiment of the present 
invention. 

FIG. 10 illustrates a GUI for a client apparatus according to a second 
embodiment of the present invention. 



WO 01/53922 PCT/US01/02241 



FIG. 1 1 illustrates a GUI for a client apparatus according to a third 
embodiment of the present invention. 

FIG. 12 illustrates a GUI for an interactive opinion research report 
generated according to one embodiment of the present invention. 
5 FIG. 12a illustrates a GUI for an interactive opinion research report 

generated according to another embodiment of the present invention. 

FIG. 13 illustrates a GUI for an opinion research summary report 
generated according to one embodiment of the present invention. 

FIG. 14 illustrates a GUI for an interactive opinion research report 
10 generated according to another embodiment of the present invention. 

FIG. 15 illustrates a GUI for a navigation guide according to one 
embodiment of the present invention. 

FIG. 16 illustrates a GUI for a navigation guide according to another 
embodiment of the present invention. 
1 5 FIG. 1 7 illustrates a GUI for an authoring environment according to one 

embodiment of the present invention. 

FIG. 18 illustrates a GUI for opinion research post analysis according to 
one embodiment of the present invention. 

Detailed Description of the Preferred Embodiments 

20 Table of Contents 



I . Overview of Real Time Polling System 

II. Client Apparatus 

A. Event Interface 

B. Polling Interface 
25 C. Input Device 

D. Synchronization and Opinion Data Transmissions 

E. Independent Media Streaming 
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III- Opinion Research Server 

A. Media Management Server 

B. Load Balancer 

C. Polling Servers 

D. Collection and Display Server 

E. Data Checkpoint and Storage 

F. User Demographic Database 

IV. Software and Hardware Embodiments 

V. Exemplary Polling Session 

VI. Opinion Research Navigation 

VII. Poll Creation and Management 
VIIL Conclusion 

I. Overview of Real Time Polling System 

FIG. 1 illustrates, according to one embodiment of the present invention, 
a block diagram of a real time polling system 100 for collecting, analyzing and 
reporting opinion research data. As shown in FIG. 1, polling system 100, in a 
representative embodiment, includes a media management server 134 in 
communication with one or more multimedia devices 146. Such multimedia 
devices include, but are not limited to, a video camera, record/playback device 
(RPD), and other devices capable of producing audio, video, text, graphics or any 
combination thereof to be sent over a network connection. RPD can be a video 
tape recorder/player (VTR), a video server, a virtual recorder, a digital audio tape 
(DAT) recorder, or any device that stores, records, generates or plays back via 
magnetic, optical, electronic, or any other storage media. Such multimedia 
devices can also include devices capable of producing video and audio from a 
video game, VCR, camcorder, laser disk, digital video disc or digital versatile 
disc (DVD) player and satellite receiver. Line 150 represents a logical 
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communication path between media management server 1 34 and the multimedia 
devices 146 listed above. 

While the above is a representative list of multimedia devices that can be 
used in the present invention, it should be understood that any other multimedia 
device, including, but not limited to, video recorders/servers, and virtual sets, 
could be easily included and would not change the scope of the invention. Any 
presently available or future developed device that is responsive to a general 
purpose interface is encompassed by the present invention. In addition, live feeds 
(such as, speeches, debates, presentations, contests, pageants, news reports, 
sporting events, and the like) from any type of source, including satellite, 
terrestrial (fiber optic, copper, coaxial, and the like), radio, microwave or any 
other form or method of transmission, can be provided in lieu of multimedia 
devices in accordance with the present invention. 

A poll/media manager 1 42 uses media management server 1 34 to conduct 
opinion research for an event, such as, speeches, television programs, 
commercials, and the like. In an embodiment, media management server 134 
displays a graphical user interface (GUI) on a display device (not shown). Line 
1 54 represents a logical communication path between media management server 
134 and the display GUI. Poll/media manager 142 utilizes the display GUI to 
program and control media management server 134. 

Media management server 134 is also in communication with time base 
generator 138 that supports time stamping. Line 152 represents a logical 
communication path between media management server 134 and time base 
generator 138. Line 174 represents a logical communication path between time 
base generator 138 and multimedia devices 146. Media management server 134 
utilizes time generator 138 to control the creation, synchronization and 
management of media streams from multimedia devices 146. The synchronized 
media streams are routed to a participant, or panel of participants, over a diverse 
computer network 110, which includes wired or wireless local area networks 
(LANs) and/or wide area networks (WANs), such as the global Internet. Line 148 
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represents a logical communication path between media management server 1 34 
andnetwork connection 1 1 0. Thus, polling system 100 utilizes computer network 
1 10 to communicate with the participants via one or more client apparatus (not 
shown). The client apparatuses enable the participants to interact with polling 
system 100. 

Polling system 100 also includes a load balancer 114, which receives 
input from all participants. Load balancer 1 14 communicates with one or more 
polling servers 118a-118n. Line 170 represents a logical communication path 
between network connection 110 and load balancer 114. Similarly, line 168 
(shown as 168a-168ninFIG. 1) represents a logical communication path between 
load balancer 1 14 and polling servers 1 18a-l 18n. After receiving input (i.e., a 
data stream containing an opinion value) from the participants, load balancer 1 14 
routes the data stream to polling servers 118a-118n. Polling servers 118a-118n 
process the data stream to statistically analyze the participants' opinions, as 
explained in detail below. Polling servers 118a-118n communicate with user 
demographic database 130, data checkpoint and storage database 126 and 
collection and display server 122. User demographic database 130 provides a 
centralized repository of demographic information on the participants. Polling 
servers 1 18a-l 18n maintain a copy of a participant's demographic records in its 
resident memory and uses the information to analyze the participant's opinion 
data. During the analysis process, the resident memory in polling servers 1 18a- 
1 18n is periodically backed-up in data checkpoint and storage database 126, as 
described in further detail below. 

Polling system 100 also includes collection and display server 122, which 
communicates with polling servers U8a-118n, data checkpoint and storage 
database 126, user demographic database 130 and media management server 134. 
Collection and display server 122 receives statistical data from polling servers 
1 18a-l 18n and aggregates the data to produce a composite result for a single 
event. Line 166 (shown as 166a-166n in FIG. 1) represents a logical 
communication path from polling servers 1 18a-l 18n to collection and display 
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server 122. In one embodiment, collection and display server 122 exchanges 
demographic data with user demographic database 130 and back-up data with 
data checkpoint and storage database 126. Lines 158 and 160 show logical 
communication paths for the demographic and back-up data, respectfully. In this 
5 embodiment, collection and display server 122 also exchanges the demographic 

and back-up data with polling servers 1 1 8a- 1 1 8n. A logical communication path 
representing the exchange of data with polling servers 1 1 8a- 1 1 8n is illustrated in 
FIG. 1 by Line 162. Collection and display server 122 and polling servers 1 18a- 
1 18n are connected to communication infrastructure 164 (e.g., communications 

10 bus, cross-over bar, or network) that provides bi-directional communications. 

Polling servers 1 1 8a- 1 18n, collection and display server 122 and media 
management server 134, each, represent one or more computers providing various 
shared resources with each other and to the other network computers. The shared 
resources include files for programs, web pages, databases and libraries; output 

15 devices, such as, printers, plotters and audio/video recorders and players; and 

communications devices, such as modems and Internet access facilities. The 
communications devices can support wired and wireless communications, 
including satellite, terrestrial (fiber optic, copper, coaxial and the like), radio, 
microwave and any other form or method of transmission. Each server is 

20 configured to support the standard Internet Protocol (IP) developed to govern 

communications over public and private Internet backbones. The protocol is 
defined in Internet Standard (STD) 5, Request for Comments (RFC) 79 1 (Internet 
Architecture Board). The servers can also support transport protocols, such as, 
Transmission Control Protocol (TCP), User Datagram Protocol (UDP) and Real 

25 Time Transport Protocol (RTP). In one embodiment, the servers use a TCP/IP 

protocol to provide communications between any two nodes on the network. Each 
server is also configured to support various operating systems, such as, 
Netware™ available from Novell®; MS-DOS®, Windows NT® and Windows® 
3.xx/95/98/2000 available from Microsoft®; Linux® available from Linux 
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Online Inc.; Solaris™ available from Sun Microsystems, Inc.; and the like as 
would be apparent to one skilled in the relevant art(s). 

II. Client Apparatus 

As discussed above, polling system 100 communicates with one or more 
client apparatuses via network connection 1 10. Each client apparatus can be a 
personal computer, personal digital assistant (PDA), telephone, television or like 
devices linked to computer network 1 10 and including a display device with the 
ability to select one or more events and indicate an opinion. The display device 
for the client apparatus provides a text or graphical user interface (GUI) and 
enables aparticipant to interactively communicate with media management server 
134 and load balancer 1 14. FIG. 2 illustrates an embodiment of client GUI 200. 

A. Event Interface 

GUI 200 can display one or more windows. In one embodiment, event 
window 214 manages the display and participant interaction with the media 
streams that are routed to the participant from media management server 134.TTie 
media streams contain the event that is being reviewed and polled by the 
participant. Event window 214 can be used to evaluate the participant's opinion 
regarding a variety of events, such as, testing of speeches, debates and 
presentations; analysis and testing of commercials and their elements; reactions 
to new product introductions; response analysis for movie and television 
programs and content; response to personalities in live or test situations; testing 
andevaluationoftestimonyandsummariesmutigauonsupporta 
the like as would be apparent to one skilled in the relevant art(s). 

Accordingly, polling system 100 can be used to conduct live and on- 
demand polls. For on-demand polling, the participant can request to view an 
event that has beendelayedor archived to storagemedia. Polling system 100 and 
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the client apparatuses can be configured to support Network 24, RealNetworks®. 

Microsoft® and Apple® applications, or other applications for downloading 

and/or streaming media as would be apparent to a person skilled in the relevant 

art(s). As such, event window 214 operates with a third party media management 
5 client to manage the display and participant interaction with the event and serves 

as the transport platform for the delivery of frame synchronization data for real 

time polling system 100. 

In one embodiment, the event is routed to the participant from media 

management server 134. In another embodiment, the event can be routed to the 
10 participant from a third party source. For example, the participant can request to 

view an event from another web site, such as streaming a live newscast from a 

news service. In this embodiment, the third party would provide the time stamped 

media frames to be synchronized with polling system 100 as discussed below. 

The event can also be contained on a storage medium located within or connected 
15 to the client apparatus. For instance, the event can be read from a hard disk drive, 

removable storage drive, removable storage interface, and the like as described 

in more detail below. 

B. Polling Interface 

GUI 200 also includes polling window 210 that provides a graphical 
20 representation of the participant's current opinion. FIG. 2 illustrates one 

embodiment of the graphical representation. In this embodiment, the participant 
can indicate an opinion by interacting with an input device (not shown) that 
moves polling indicator 218 to designate a discrete value between 0 and 100. "0" 
represents a strongly unfavorable response, "50" represents a neutral response, 
25 and "100" represents a strongly favorable response. 

FIG. 2a and FIG. 2b illustrate two additional embodiments of the 
graphical representation of the participant's current opinion. In FIG. 2a, the input 
device permits the participant to move an indicator mark 240 along horizontal bar 
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244 which interacts with polling indicator 218 to signal a "thumbs-up" or 
"thumbs-down" response. As such, if indicator mark 240 is moved all the way to 
the right, polling indicator 218 would signal a thumbs-up to indicate a strongly 
favorable response. If indicator mark 240 is moved all the way to the left, polling 
indicator 218 would signal a thumbs-down to indicate a strongly unfavorable 
response. At any location between the left-most and right-most positions, the 
participant can signal an intermediate level of satisfaction. 

Referring to FIG. 2b, the input device permits the participant to move 
indicator mark 240 along horizontal bar 244 to instruct polling indicator 218 to 
signal a discrete value between 1 and 10. "1" represents a strongly unfavorable 
response, "5" represents a neutral response, and "10" represents a strongly 
favorable response. As can be seen, the graphical representation of the 
participant's opinion can be illustrated as a dial or other graphical metaphors as 
would be apparent to one skilled in the relevant art(s). It can provide discrete 
values as shown in FIGs. 2 and 2b or analog values as shown in FIG. 2a. 

As described above, polling indicator 218 provides real time indication 
of the participant's current opinion regarding the event that is being viewed on 
eventwindow214. Panel value 222 provides real time communication of current 
opinions for all participants. The client apparatus includes a graphics application 
program interface (API) that interfaces between the media management server 
134 and polling window 210. The API provides for customization of the display 
attributes without effecting changes to the underlying data management 
structures. As such, collection and display server 1 22 tabulates the opinions of all 
participants on the panel to generate a composite result and routes this 
infomauon(viamedia managements 

composite result is displayed as panel value 222, as discussed in greater detail 
below. 

Various embodiments of the GUI for the client apparatus, including 
polling window 210, can easily be included with the methods and systems of the 
present invention. Any currently available or future developed user interface 
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capable of collecting and sending opinion values to polling server 1 1 8a- 1 1 8n is 
considered to be within the scope of the present invention. Accordingly, FIG. 10 
illustrates another embodiment of a GUI for the client apparatus (shown as GUI 
1000). As shown, GUI 1000 includes event window 214, indicator bar 244, 
polling indicator mark 1018, indicator scale 1048 and script window 1052. As 
discussed in reference to FIG. 2, event window 214 provides a forum for 
presenting a video, audio, multimedia or like forms of the event being polled. 

Adjacent to event window 214, script window 1052 receives captions or 
text from the polled event. Thus, in one embodiment, a complete transcript of the 
polled event is synchronized with the media streams and displayed in script 
window 1052. In one embodiment, the event is provided with captions or text 
from the source. In another embodiment, well-known captioning and stenographic 
technologies are used to produce the captions or text and synchronize the data 
with the media stream. The methods and systems of the present invention 
supports both live and delayed captioning for the events. Accordingly, in one 
embodiment, real time captions or text can be created and matched to the audio 
track as live or delayed media streams are sent to the client apparatuses. In 
another embodiment, pre-recorded media streams can be processed to generate 
the captions or text that are timed and matched to the media streams. 

As discussed, the captions are displayed in script window 1052. However, 
in one embodiment, event window 214 is configured to support embedded 
captioning in media streams. Therefore, in this embodiment, the captions are 
located in image displayed in event window 2 14 in lieu of, or in addition, to script 
window 1052. 

Referring to FIG. 10, positioned beneath event window 214 is indicator 
bar 244. A participant moves or slides polling indicator mark 1018 along the axis 
of indicator bar 244 to specify an opinion value for the event presented in event 
window 214. As shown in FIG. 10, indicator bar 244 can be illustrated as a line 
segment in a horizontal position on GUI 1000. As would be apparent to one 
skilled in the relevant art(s), indicator bar 244 can vary in shape, size and layout. 
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As such, in other embodiments, indicator bar 244 can be depicted in a vertical 
position, displayed in the form of an arc or circle, shaped like a polygon, and the 
like. As shown in FIG. 1 0, indicator bar 244 can be illustrated in two dimensional 
space. Alternatively, indicator bar 244 can be displayed with three or more 
coordinate dimensions. 

Indicator scale 1048 provides an array of possible opinion values to be 
selected by the participant As shown, theopinion values are expressed as discrete 
values ranging from 0-100. However, as discussed with reference to FIGs. 2-2b, 
indicator scale 1048 can be expressed in various discrete or analog schemes,' 
including metaphorical references, such as facial or hand expressions, directional 
signals, arrows, plus or minus signs, other contrasting graphical icons, and the like. 
The textual labeling for indicator scale 1048 can express categories, such as, 
strongly agree, agree, neutral, disagree and strongly disagree, and the like. As 
shown in FIG. 10, the textual labeling for indicator scale 1048 can always be 
present for ease of reference. In another embodiment, the textual labeling can be 
formatted to change intensity or color as polling indicator mark 101 8 traverses 
indicator bar 244. In another embodiment, indicator scale 1048 can be blank, and 
part or all of the textual labeling can be formatted to appear only as polling 
indicator mark 1 01 8 is moved. 

Referring to FIG. 10, a display area is positioned on topofpolling indicator 
mark 1018 to display an opinion value. Thus, as polling indicator mark 1018 
traverses indicator bar 244, the display area would dynamically reveal a specific 
opinion value corresponding to indicator scale 1 048 . As discussed with reference 
to indicator scale 1048, the display area can also produce discrete values, or 
metaphorical expressions and the like. In one embodiment, indicator scale 1018 
is blank, and the opinion value is only revealed on the display area for polling 
indicator mark 1 0 1 8 . In another embodiment, the opinion value is not revealed on 
the display area for polling indicator mark 1018, but is displayed only by indicator 
scale 1018. Although polling indicator mark 1018 is illustrated in GUI 1000 asa 
slider,oneskilledmthe^^^^ 
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for the opinion responses. For example, instead of including indicator bar 244, 
GUI 1000, in another embodiment, can display an object similar to polling 
indicator mark 1018 with a display area. The participant could use an input device 
to designate discrete or metaphorical expressions on the display area. 

Thus, as can be seen, a variety of graphical metaphors can be used to 
display the participant's opinion value that is forwarded to polling servers 1 18a- 
118n. For example, FIG. 11 illustrates another GUI for displaying both the 
participant's opinion and a composite opinion for an entire panel. GUI 1100 
includes a vertically positioned indicator bar 244. No textual labeling is presented 
along the axis of indicator bar 244. However, the endpoints are labeled "positive" 
and "negative." As discussed above, the endpoints can be other labels indicating 
a positive or negative opinion, including "vote for" or "vote against," "yes" or "no," 
and the like. Polling indicator mark 1018 includes a dynamically changing display 
area that reveals discrete opinion values. Display region 1 1 04 exhibits a graphical 
representation of the participant and panel's opinion values. Participant legend 
1 156 and panel legend 1 160 provides a key or legend for distinguishing between 
the participant and panel's opinion values, respectively. In one embodiment, the 
opinion values can be presented in display region 1 104 as a function of time to 
show the moment-to-moment responses from the participant and the real time 
aggregate responses from all participants. As a result, the participant can use 
display region 1 104 to compare the participant's response (shown by participant 
legend 1 1 56) with the rest of the panel (shown by panel legend 1 1 60). To illustrate 
the participant and panel's responses, the opinion values can be plotted as a 
function of time on a graph. In another embodiment, the participant and panel's 
responses can be depicted on bar charts, histograms, pie charts, and the like, as 
would be apparent to one skilled in the relevant art(s). In another embodiment, an 
overlay of the participant's responses can be positioned over the panel's response. 
To more easily view the two images, readily distinguishable colors, line styles, and 
the like can be used. As would be apparent to one skilled in the relevant art(s), the 
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illustrations in display region 1104 can be displayed in two, three or more 
dimensions. 

The above-mentioned GUI embodiments for the client apparatus have been 
described in reference to data acquisition. However, the present invention also 
encompass the use of the above GUIs to replay polled events and/or review the 
polled results. In other words, after an event has been polled, a pollster or other 
individual can download the results for post analysis. Referring back to FIG. 2, 
media streams from the event can be replayed on event window 214. Polling 
window 210 can be used to present a graphical representation of the opinion 
responses. Accordingly, polling indicator 2 1 8 would move synchronously with the 
media frames. In one embodiment, the opinion dial in polling window 210 can be 
replaced with a graph, bar chart, pie chart and the like, (comparable to panel graph 
1160 shown in FIG. 11) to display the composite opinion values. Similarly, 
referring back to FIG. 10, polling indicator mark 1018 and/or indicator scale 1048 
can be configured to reveal the composite opinion value for each media frame as 
it is displayed in event window 214. 

As discussed in detail below, collection and display server 122 can also 
produce electronic and paper reports for each polled event. Such reports can also 
be displayed on the GUI for the client apparatus. FIG. 1 2 illustrates, according to 
one embodiment, GUI 1200 for displaying an interactive opinion research report 
GUI 1200 includes response graph 1204 and script window 1052. Response graph 
1204 displays a graphical representation of the composite opinion values as a 
function of time. As can be seen, in one embodiment, the composite opinion 
values are plotted to create a graph, but other formats such as, charts, histograms, 
and the like can be used, as would be apparent to one skilled in the relevant art(s). 
In this embodiment, the value for the opinion responses are referenced with respect 
to the ordinate and time (e.g., seconds, milliseconds, minutes and the like) is 
referenced as the abscissa. Alternatively, the coordinate axises could be switched 
with the opinion responses being the abscissa and time being the ordinate. 
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The time axis, shown in FIG. 12 as the abscissa, car. provide an index into 
the media streams of the event. As shown, the time axis contains a plurality of time 
marks that represent links or hyperlinks to the individual frames from the media 
streams. As such, each time mark is cross-referenced to the time stamps in the 
media event. Activation of one of the time marks would, as a result, cause the 
captions or text from the video, audio or the like to be displayed in script window 
1 052. The captions would be displayed at the precise point cross-referenced to the 
activated time stamp. 

In another embodiment, the GUI 1200 can include a display area (not 
shown) that is comparable to event window 214. In this embodiment, activation 
of the time mark would cause the actual video, audio and the like of the polled 
event to be viewed on GUI 1200. If the polled event is visual (i.e., video, images 
and the like), activating the time mark can produce either a static display of 
individual frames or a dynamic media stream. The media feed can be in addition 
to, or in lieu of, the captioned data shown in script window 1052. 

The graph line illustrated in response graph 1 204 can also provide an index 
into the media streams. In one embodiment, operating an input device (discussed 
below) to activate any point along the graph line can, as a result, activate an 
associated time mark to obtain the captions or media streams of the polled event 
In one embodiment, activating a point on the graph line can cause the associated 
time mark to change colors or reveal a discrete time value. In another embodiment, 
the points along the graph line are direct links into media streams for polled event, 
without regard to the time marks. 

As shown in FIG. 12, the graph lines are in different colors. Different 
colors or line styles can be used to designate the opinion responses from different 
groups, individuals, polled events, filtered views, and the like. For example, a 
participant can request a summary report (i.e., GUI 1200) to compare the 
participant's responses to those of the entire panel. Thus, as shown, the 
participant's responses would be represented in one color and the panel's 
composite results are shown in a second color. The two graphs would be 
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superimposed over each other for comparison. The participant can also request a 
comparison with different demographic groups. For instance, participant's 
responses can be graphed and compared with people of a certain age, income, 
educational level, geographic location, religion and the like. As such, multiple 
graphs in different colors or line styles can be displayed and compared in response 
graph 1204. Response graph 1204 can also be used to compared multiple polled 
event. For instance, one can request to compare a panel's opinion of an specific 
episode of a television sitcom that was broadcasted on different nights or time 
segments. The results from each broadcast could be graphed in different colors and 
overlaid on each other for cross-comparison. As would be apparent to one skilled 
in the relevant art(s), other embodiments of response graph 1204 can be used to 
evaluate one or more opinion polls. 

As shown in FIG. 12, response graph 1204 and script window 1052 can be 
placed in HTML format to be displayed on a browser. Alternatively, the data can 
be formatted to be printed as a paper report, as would be apparent to one skilled 
in the relevant art(s). 

FIG. 12a illustrates GUI 1200A another embodiment of an interactive 
opinion research report GUI 1200A includes a plurality of speaker icons 1208a- 
1 208n, each associated with an individual participating in the event. For example, 
if the event is a political debate among seven candidates, each candidate would 
have a separate speaker icon 1208a-1208n. Each speaker icon 1208a-1208n is 
coupled to its respective speaker bar 1244a-1244n. On top of each speaker bar 
1244a-1244n is an indicator mark 1218a-1218n. 

moneembodimentoperatmganmpm 
one of the speaker icons 1208a-1208n would automatically highlight (i.e., 
changing color or intensity) that portion the time axis cross-indexed to the media 
frames for the associated speaker. The corresponding portion of the graph lines 
shown in response graph 1204 can also be highlighted to Ulustrate the composite 
opinion values when the speaker icon 1208a-1208n is activated. In one 
embodiment, activatingaspeakericonl208a-1208ncanalso automatically trigger 
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the time marks to initiate the associated captions in script window 1052 and/or 
static or dynamic displays of the speaker's media frames in event window 2 14 (not 
shown in FIG. 1 2a). Alternatively, the user can activate the highlighted time marks 
by, for example, clicking on the time marks to start the media feed to event 
window 214. 

Referring to FIG. 12a, speaker bars 1244a-1244n and indicator marks 
1218a-1218n enable a user to manipulate the media streams cross-linked to the 
respective speaker icon 1208a-1208n. As discussed, activation of a speaker icon 
1208a-1208n, in one embodiment, would highlight the cross-indexed time axis. 
Moving an indicator mark 1 2 1 8a- 1 2 1 8 along the spine of its respective speaker bar 
1244 would, in turn, activate the cross-indexed time mark(s) on the time axis. 
Thus, indicator marks 1 2 1 8a- 1 2 1 8n are graphical controls for the content of script 
window 1052 and/or event window 214 (if included). 

Although speaker icons 1208a-1208n are described with reference to 
individuals engaging in the media events, it would be apparent to one skilled in the 
relevant art(s) that speaker icons 1208a-1208n can be used to create a cross-index 
to various components, features or aspects of the event. For example, each speaker 
icon 1208a-1208n can be used to designate specific issues or topics in a 
presentation, scenes within a broadcast production, segments in a news program, 
tracks from an audio recording, performances within a live concert, and the like. 

Referring to FIG. 13, GUI 1300 illustrates, according to one embodiment, 
an opinion research summary reported generated by collection and display server 
122 (described in detail below). Unlike GUIs 1200 and 1200a, the summary report 
presented in GUI 1300 is not interactive. Rather, GUI 1300 provides a snapshot 
of the composite responses from the polled event GUI 1 300 includes a composite 
region 1308 and filtered region 13 12. In this embodiment, the composite opinion 
value is computed for the panel and summarized by five categories: strongly agree, 
somewhat agree, undecided, somewhat disagree and strongly disagree. The five 
categories correspond to the indicator scale on the polling client (not shown). 
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Filtered region 1312 displays filtered views of the composite results. As 
shown, the opinion responses can be demographically filtered by political 
affiliation (e.g., republican, democrat or independent), gender and age. As 
discussed below, other filtering criteria (e.g., income, marital status, religion, 
nationality.race, profession, address, psychographic or behavioral patterns, andthe 
like) can be used to process, analyze and display the opinion research data as 
would be apparent to one skilled in the relevant art(s). 

Referring to FIG. 14, GUI 1400 illustrates another embodiment of an 
opinionresearch summary reported generated by collection and display server 122 
(described in detail below). GUI 1400 includes a response graph 1404, primary 
region 1408 and a plurality of subordinate regions 1416a-1416n. Similar to 
response graph 1204 (described with reference to FIG. 12), response graph 1404 
displays a graphical representation of the composite opinion values as a function 
of time and, in one embodiment, can display multiple graphs in different colors or 
line styles for comparable studies between different groups, polls and the like. 
Primary region 1408 summarizes the opinion data for the entire polled event, at 
some point in time. Primary region 1408 includes a composite value for the panel 
in addition to filtered results by gender, political affiliation and a specific age 
range. Subordinate regions 14I6a-1416n provide a breakdown of the composite 
result by topics or issues presented in the polled event. Each subordinate region 
1416a-1416n has adistinct topic or issue. As can be seen, each subordinate region 
1416a-1416n includes a composite value per topic with filtered results by 
demographic category. As would be apparent to one skilled in the relevant art(s), 
the summary reports can be presented in other layouts and forms, which are 
deemed to be within the scope of the present invention. 

FIG. 18 illustrates another embodiment of GUI 1800 for post opinion 
research analysis. GUI 1800 permits a research to readily filter and plot plural 
demographic data. Aresearcherwould use controls 1802 to plots results on 1804. 

In one embodiment, the graphical representations (shown in 1 156, 1 160, 
1204, 1404 and the like) of the opinion values are calculated on collection and 



RNSDOCID: <WO 0153922A2J_> 



WO 01/53922 



PCT/US01/02241 



-22- 



display server 122 and sent to the client GUIs in real time or on-demand. In 
another embodiment, all or part of the graphical representation can be calculated 
by a resident application on the client apparatuses. 

C. Input Device 

5 The input device can be a mouse, mouse wheel, joystick, rudder pedals, 

keyboard, touch screen, microphone, joystick, stylus, light pen, voice recognition 
unit or any other type of peripheral unit. In one embodiment, the client apparatuses 
are configurable to receive verbal commands to execute various tasks during the 
polling or post analysis sessions (e.g., interactive voice response (IVR)). Using a 
10 rule-based expert system, a processing unit enables a voice recognition unit to 

identify the voice of the user. The aforementioned is a representative list of input 
devices that can be used with the present invention, it should be understood that 
any other type of input device capable of measuring or monitoring the reactions or 
responses of an individual to the event, including, but not limited to, foot pedals, 
15 devices formeasuring respiration, blinking, pulse orheartrate or other biometrics, 

or the like as would be apparent to one skilled in the relevant art(s) could be easily 
included and would not change the scope of the invention. For example, a poll 
participant can attach a biometric device to a hand or other body part. As the 
participant moves the hand through space, this movement can be translated and 
20 feed to the client apparatus to cause a corresponding movement in the polling 

indicator. Thus, any presently available or future developed device that is 
responsive to ageneral purpose interface is encompassed by the present invention. 

A polling API interfaces between the client GUIs and the input device to 
support real time visual feedback to polling window 210 or the other graphical 
metaphors and graphical controls mentioned above. The polling API provides for 
the use and change of the appearance and methodology of the actual "dial control" 
interface (i.e., polling window 210) and other graphics without affecting the 
underlying communications and management technology. 
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D. Synchronization and Opinion Data Trans 



missions 



The system and method of the present invention generates a discrete value 
for the participant's current opinion, synchronizes the media streams with the 
opinion value and routes the synchronized data to load balancer 114. In an 
embodiment of the present invention, a Java applet or JavaScript controls the 
creation, synchronization, management and routing of the opinion data with the 
media steams. In another embodiment, other programrning languages, such as 
JavaScript, C, C++, and the like, can be implemented and are considered to be 
within the scope of the present invention. Each time the participant indicates a 
current opinion, the polling API establishes a quantitative value for the opinion 
depicted by polling indicator 218. A time stamp is also generated for the opinion 
value. For example, referring to FIG. 2, if the participant operates the input device 
to indicate an opinion value of "50" at time "22:24:02," the value "50" and time 
stamp "22:24.02" would be recorded. The opinion value time stamp is used to 
synchronize the opinion value with the media frame the participant was viewing 
at the time the opinion value was created. 

In one embodiment, each media frame includes a time stamp (e.g., time or 
frame code) to identify each frame within the media streams. In this instance, the 
media frame time stamp is used to synchronize the opinion value with the media 
streams. In another embodiment, the client apparatus can generate the time stamps 
used to mark and identify each frame. As would be apparent to one skilled in the 
relevant art(s), any similar technique for identifying individual or a group of 
individual frames within a media stream can be integrated with the present 
invention to synchronize the opinion value with the media streams. 

The Javaappletcreatesadatapacketcontaimngtheopinion value and time 
stamp and routes the data packet to load balancer 1 14. In one embodiment, a data 
packet includes the following frames: 



(UserJD, PollJD, Timestamp, Value) 
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where "User_ID M is a unique user identification field that identifies the participant 
to user demographic database 130. "Poll_ID" is a unique identifier that identifies 
the poll associated with the opinion data. This allows polling system 100 to 
support multiple simultaneous events, as well as multiple simultaneous polls for 
the same event. "Value" is the response indication value specified in a range 0 — 
256 (one byte) and is the captured value of the participant's response or opinion 
regarding the event "Timestamp" is a time or frame indicator that synchronizes 
the opinion value data with the routed media streams, i.e., client-side audio and 
video. Timestamp is used to synchronize the opinion value data with the routed 
media streams. Time stamping is a key element in providing synchronization over 
the Internet where latency would vary with each participant. Polling system 100 
uses the time stamps to properly interpret the data. 

Another embodiment of the data packet includes: 

15 (User_ID, PollID, ValueTimestamp, Value, MediaJTimestamp) 

where "ValueJTimestamp" is the time indicator that captures the time the Value 
was created. "MediaJTimestamp" is a time or frame indicator that identifies a 
specific frame within the routed media streams. To support time stamping, the 
method and system of the present invention provides an absolute way of utilizing 

20 the client apparatus' computer clock and an absolute standard to ascertain a unique 

time base for each participant. In one embodiment, the media management server 
1 34 sends instructions to all client apparatuses to synchronize the computer clocks 
for all participants. An absolute time scheme, such as Greenwich Mean Time, can 
be used as the time standard. Therefore, in this embodiment, the time stamps 

25 would be identical for all participants. Thus, media management server 134 has the 

ability to set client clocks remotely. In this instance, the Java applet would create 
a dialog box prompting the participant's permission to reset the clock. 

As discussed, the data packets are created and routed to load balancer 1 1 4. 
The frequency of the data transmission to load balancer 1 1 4 is controlled by media 
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management server 134. Thus, based on the operational mode established by 
media management server 134, the client apparatus transmits the data packets, 
including the time stamps, at a periodically scheduled rate or on an event-driven 
basis. As discussed above, the time stamping keeps all opinion data synchronized 
to the event. This transmission strategy overcomes problems due to variable delays 
and data loss. For instance, since each participant may experience the media 
streams at a different time due to latency, a media frame viewed by one participant 
may appear seconds apart for those viewed by other participants. Time stamping 
enables polling system 1 00 to keep the composite opinion value synchronized with 
the media streams. Accordingly, the present invention provides real time 
measurement of network, including Internet, latency as part of time 
synchronization. 

E. Independent Media Streaming 

In an alternate embodiment of the present invention, polling system 100 
can be configured to support stand-alone, independent media streams. Independent 
media streams represent an event that is provided by a third party source, but is not 
routed to the client apparatus. Independent media streams can also be provided by 
multimedia devices 146 to a device other than the client apparatus. Such media 
streams can include, but is not limited to, a television show broadcasted on the 
participant's television, world-premiere televised broadcast of a new music video, 
musical program or song broadcasted on a radio, motion picture shown on a 
projector screen at a film festival, musical composition recorded on a compact disc 
and played over speakers in a theater, a recording of an awards ceremony or 
theatrical performance wherein the recording is time stamped and streamed in real 
time, delayed or on-demand to other locations, and the like as would be apparent 
to one skilled in the relevant art(s). 

Referring again to FIG. 2, GUI 200 can contain an icon (not shown) for 
polling independent media streams. Once the icon has been selected, GUI 200 
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permits the participant to interact with polling window 2 1 0 : except event window 
214 would not be present. The participant receives and evaluates the independent 
media streams as they are routed to or from a stand-alone apparatus. The 
stand-alone apparatus includes a television, radio, telephone, PDA, personal 
computer, theater screen, film projector, video server, other multimedia devices 
or similar apparatus configured to receive media signals and convert the signals 
to a medium capable of being processed by the participant. In one embodiment, the 
stand-alone apparatus can be a multimedia device used to record the event, such 
as a live concert, play, public debate, speech, presentation, sporting event, news 
reports and the like. In this embodiment, the participant would watch the event as 
it is being recorded and time stamped by the stand-alone apparatus. The stand- 
alone apparatus of this embodiment would route the independent media streams 
to media management server 134. 

As intimated, the stand-alone apparatus is separate and independent from 
the polling client apparatus. As such, the participant can interact with polling 
system 100 to provide opinion responses on the client apparatus, e.g., a PDA, 
while viewing or listening to media streams from a stand-alone apparatus, e.g., a 
radio, or while viewing a live event that is being recorded by a stand-alone 
apparatus, e.g. video camera. 

In an embodiment of the present invention, the individual media frames 
within the independent media streams contain time stamps (e.g., time or frame 
codes) provided by the third party source, time base generator 138 or stand-alone 
apparatus, as appropriate. The time stamps provide an absolute time standard that 
enables polling system 100 to synchronize the independent media frames with the 
opinion data generated by GUI 200. In one embodiment, media management 
server 134 receives broadcast signals from the third party source to indicate which 
media frames are being provided to the participant. As such, media management 
server 134 would be able to track or count the media frames as they are being 
delivered to the participant Accordingly, when collection and display server 122 
receives the time stamped opinion data from the participant (as discussed below), 
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the opinion data can be correlated with the independent media streams. For 
example, at time 10:55:18, the third party source can inform media management 
server 134 that UserJD "58679™ is receiving frames 20:10:18 and 20:10:19. 
Therefore, if collection and display server 122 receives opinion Value "70" at 
Timestamp "10:55:18," polling system 100 would know that this opinion Value 
relates to frames 20: 1 0: 1 8 and 20: 1 0: 1 9. 

In oneembodiment,mediamanagement server 134 synchronizes the clock 
in the client apparatus with time base generator 138. Time base generator 138 
would be synchronized with the stand-alone apparatus. For example, media 
management server 1 34 can transmit signals to reset the client clock, as discussed 
below, or alternatively, it can reset a resident time keeping unit within time base 
generator 1 38 to synchronize the resident time keeping unit with the client clock. 
Thus, when collection and display server 122 receives the time stamped opinion 
data (as discussed below), the time stamps would be synchronized with those 
generated by the time keeping unit within time base generator 138. As a result, 
polling system 100 would be able to correlate the opinion data with the media 
frames within the independent media streams with marginal error. 

In another embodiment, a bridging device (not shown) is used as an 
interface between the polling client apparatus and the stand-alone apparatus. The 
bridging device counts the frames or reads the time stamps from the independent 
media streams as they are being delivered to or sent from the stand-alone 
apparatus. The bridging device routes these signals to the polling client, where a 
Java application uses the data to time stamp the opinion data as discussed above 
in reference to the polling interface. The time stamped opinion data is, therefore, 
routed to load balancer 1 14 where it is processed as discussed below. 
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III. Opinion Research Server 

A. Media Management Server 

FIG. 3 illustrates, according to one embodiment of the present invention, 
5 a block diagram of media management server 134. FIG. 3 is a conceptual 

illustration of media management server 1 34 that allows an easy explanation of the 
present invention. That is, one or more of the blocks can be performed by the same 
piece of hardware or module of software. It should also be understood that 
embodiments of the present invention can be implemented in hardware, software, 

10 or a combination thereof. In such an embodiment, the various components and 

steps would be implemented in hardware and/or software to perform the functions 
of the present invention. 

As shown in FIG. 3, media management server 134 in a representative 
embodiment, includes a registering unit 310, polling management unit 314, 

15 formatting unit 318, media routing unit 322 and memory unit 326. Registering unit 

310 communicates with a plurality of client apparatuses and participants over 
network connection 1 10. A participant registers for a polling event by sending a 
registration request to registering unit 310. Registering unit 310 verifies the 
participant's registration information with the participant's existing profile, or 

20 establishes a new profile, as appropriate, by communicating with user 

demographic database 130. In one embodiment of the present invention, a copy of 
user demographic database 130 is located in memory unit 326 for efficient 
processing. As shown in FIG. 3, registering unit 310 processes the participant's 
registration request by contacting memory unit 326. Memory unit 326 maintains 

25 coherency with user demographic database 130, as discussed below. In another 

embodiment of the present invention, registering unit 3 10 has a direct connection 
with user demographic database 130. Accordingly, in this embodiment, user 
demographic database 130 is automatically updated with any changes 
communicated by the participant in the registration request. 
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Polling management unit 314 exchanges signals with registering unit 310 
and formatting unit 3 1 8. Referring to FIG. 3, registering unit 3 10 sends a signal to 
polling management unit 3 14 to indicate which participants have been properly 
registered to participate in the polling event. Registering unit 310 is also 
configured to transmit participant requests to terminate a polling session. Upon 
receipt of the registration or termination signal, polling management unit 314 
routes the signal to formatting unit 3 1 8 to initialize and manage, or terminate, the 
polling session, as appropriate. Polling management unit 314 can also terminate 
a designated or all polling sessions independent of participant input. The 
initialization or tennination signal is routed to formatting unit 3 18 to identify the 
network address (i.e., IP address) of the registered participants. 

Formatting unit 3 1 8 provides the initialization instructions for creating the 
polling GUIs (e.g., GUI 200, GUI 1000, GUI 1100 and the like) for client 
apparatuses. In one embodiment, the instructions are included within a Java applet 
that is routed to the client apparatuses. Regardless of the design for the polling 
indicator, indicator bar and the like, formatting unit 3 1 8 places the graphics in a 
certain format and passes the physical size of the graphic images and name of file 
containing the images to the Java applet to manage them. 

Polling management unit 314 also manages the ongoing operations of 
polling system 100. Polling management unit 314 provides commands to 
formatting unit 3 1 8 to establish the operational modes for each polling session. For 
example, polling management unit 314 transmits signals to establish the 
synchronization scheme for the polling sessions. As discussed above, these 
instructions determine whether the client apparatuses would time stamp the 
opinion value data with, for example, time codes or frame codes. The time stamps 
are subsequently used to synchronize the opinion value with the media streams. 

These synchronization instructions can also request the client apparatuses 
to reset the computer clocks to an absolute time scheme, such as Greenwich Mean 
Time. Accordingly, polling management unit 314 has the capability to remotely 
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set the client clocks to synchronize the time stamps (i.e., time codes) for all 
participants. 

Formatting unit 3 1 8 embeds the synchronization instructions into the data 
packets routed to the Java applet in the client apparatuses. Polling management 
unit 3 14 also provides instructions to formatting unit 3 1 8 to synchronize the media 
streams from multimedia devices 146 with the time stamps from time base 
generator 138, as discussed below. 

A second operational mode set by polling management unit 314 is the 
transmission rate. Polling management unit 314 determines the frequency for 
updating the composite result (shown in FIG. 2 as panel value 222), as well as 
sending any graphical data related to the composite result for each polling session. 
These instructions are routed and implemented by formatting unit 318. 

Polling management unit 314 also determines the frequency for reporting 
each individual participant's value data to load balancer 1 14. The transmission 
instructions are routed to formatting unit 318 which, in turns, embeds the 
transmission instructions into the data packets routed to the Java applet in the 
client apparatuses. 

For instance, the reporting frequency can be event-driven or based on a 
periodic sampling. If event driven, a participant's value data is routed to load 
balancer 114 each time the participant indicates or changes the opinion value 
(shown in FIG. 2 as polling indicator 218). If based on periodic sampling, the 
value data would be stored and reported at a designated time. For example, the 
Java applet can be instructed to transmit a specified number of data frames per unit 
of time. 

Another operational mode determined by polling management unit 3 14 is 
automatic reset. If selected, polling management unit 314 would instruct 
formatting unit 3 1 8 to route resetting instructions to the Java applet in the client 
apparatus. The Java applet would be instructed to automatically return polling 
indicator 218 to, for example, the "50%" mark if no response has been indicated 
for a designated period of time. 
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The Java applet can also be instructed to place the client apparatus in a 
"time-out" state. If selected, after a specified period of time has elapsed since a 
change in opinion value (shown in FIG. 2 as polling indicator 218), polling 
management unit 3 1 4 would terminate or temporarily suspend the polling session 
and instruct the participant's browser to download another web page. If thepolling 
session is temporarily suspended, the polling session can be terminated after a 
specified period of time has elapsed without participant interaction. The Java 
applet canbe instructed to warn the participant by generating a dialog box prior to 
terminating or temporarily suspending the polling session. The time-out period for 
termination and suspension can be adjusted by poll/media manager 142. 

In one embodiment, GUI 200 can include an icon or pull-down option (not 
shown) that enables the participant to select the time-out operational mode. The 
time-out state can be activated after the elapse of a participant-designated period 
of time or immediately upon activation. This option would enable the participant 
toplac^mepolImgsessioninaVuse^ 



activities. 



While the above is a representative list of operational modes that can be 
used with the present invention, it should be understood that any other method for 
managing the operations of the polling session, including interactions with the 
client apparatus, as would be apparent to one skilled in the relevant art(s), can be 
easily included and would not change the scope of the invention. 

Media management server 134 also includes formatting unit 318 that 
exchanges signals with one or more multimedia devices 146, time base generator 
138, collection and display server 122, polling management unit 314, and media 
routing unit 322. As instructed by polling management unit 3 14, formatting unit 
318 contacts and receives composite opinion data from collection and display 
server 122. A data packet wmtaining the composite data is subsequently 
transmitted to media routing unit 322. The frequency for receiving and 
transmitting the composite data is determined by polling management unit 3 1 4, as 
discussed above. 
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As instructed by polling management unit 314, formatting unit 318 
communicates with a plurality of multimedia devices 146 by sending media 
management commands to receive information from multimedia devices 146. 
Media management server 134, therefore, provides a means for centrally 
controlling each of the multimedia devices 146. For example, formatting unit 3 1 8 
can instruct one or more multimedia devices 146 to send a stream of media data 
across network 110. 

Formatting unit 318 also sends commands to time base generator 138. In 
response to the commands, an output data stream with synchronization data is sent 
to the media management server 1 34. In one embodiment, the synchronization data 
is based on an absolute time scheme, such as Greenwich Mean Time. Formatting 
unit 318 integrates the synchronization data with the media streams from 
multimedia devices 146 to create synchronized media streams. Alternatively time 
base generator 138 can send the synchronization data directly to multimedia 
devices 146 where synchronized media streams are generated and routed to 
formatting unit 3 1 8. Data packets containing the synchronized media streams are 
then transmitted to the media routing unit 322. 

Formatting unit 3 18 can also configure the media streams to make them 
compatible with the client apparatuses. This can be implemented by utilizing the 
registration data routed to registering unit 310 by the participant, that, in turn, is 
forwarded to the polling management unit 3 1 4 and finally to formatting unit 318. 
For example, the media streams can be formatted to support Apple®, Microsoft®, 
or RealNetworks® multimedia applications; Netscape® and Microsoft® browsers 
capable of handling Java 1 . 1 or higher; Windows®, Novell® or Linux® operating 
systems; Hypertext Markup Language (HTML) or Synchronized Multimedia 
Integration Language (SMIL) languages; or the like as would be apparent to one 
skilled in the relevant art(s). 

As discussed, formatting unit 318 utilizes can use a combination of 
technologies, such as HTML, SMIL or a combination of both accessed under an 
abstraction layer, to coordinate the media streams for display on the client 
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apparatuses. By using a single time line, all media on a page can be properly time 
coordinated and synchronized. Accordingly, polling system 100 supports 
multimedia standards developed by Microsoft® technologies, such as, Windows® 
Media™ Player; RealAudio® and RealVideo® formats from RealNetworks® Inc.; 
or the like as would be apparent to one skilled in the relevant art(s). In an 
embodiment, SMIL or a created abstraction of SMIL, is used to provide 
compatibility with both RealNetworks® and Microsoft® technologies. Elements 
of media management server 134 can be adopted directly from the client 
multimedia technologies, such as Windows® Media™ tool offerings. 

Media management server 1 34 also includes media routing unit 322. Media 
routing unit 322 receives the data packets generated and formatted by formatting 
unit 318. The data packets includes the synchronized media stream representing 
the event. The data packets also include composite opinion data, including 
graphics, created by collection and display server 122 and formatted by formatting 
unit 318. Reading the network address, i.e., IP addresses, in the header frames, 
routing unit 322 transmits the data packets to the designated participant over 
network connection 110. 

As previously discussed in reference to FIG. 1, poll/media manager 142 
uses media management server 134 to conduct opinion research for an event. In 
an embodiment, media management server 134 displays a GUI on a display device 
(not shown). The display GUI provides graphical controls corresponding to the 
multimedia devices 146. Poll/media manager 142 can use a keyboard, mouse, or 
other input devices to interact with the media management server 134. As 
described in reference to the client input device, it should be understood that any 
other type of input device, including, but not limited to, a touch screen, voice 
recognition unit, microphone, stylus, light pen, or the like as would be apparent to 
one skilled in the relevant art(s) could be easily included and would not change the 
scope of the present invention. 

In response to poll/media manager 142 activating a graphical control from 
display GUI, media management server 134 transmits a media management 
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command to the multimedia device 146 corresponding to the activated graphical 
control. In this manner, poll/media manager 142 centrally controls the operation 
of each of the multimedia devices 146. The display GUI also permits poll/media 
manager 1 42 to assemble the content of the events, establish the operational modes 
for polling management unit 314, develop formatting and configuration 
instructions for formatting unit 318, reconfigure or alter the records in user 
demographic database 130, and provide similar commands and information to 
control polling system 100 as would be apparent to one skilled in the relevant 
art(s). 

B. Load Balancer 

FIG. 4 illustrates, according to one embodiment of the present invention, 
a block diagram of load balancer 1 14. FIG. 4 is a conceptual illustration of load 
balancer 1 14 that allows an easy explanation of the present invention. That is, one 
or more of the blocks can be performed by the same piece of hardware or module 
of software. It should also be understood that embodiments of the present 
invention can be implemented in hardware, software, or a combination thereof. In 
such an embodiment, the various components and steps would be implemented in 
hardware and/or software to perform the functions of the present invention. 

As shown in FIG. 4, load balancer 1 14 in a representative embodiment, 
includes a balancing queue 410 and balancing routing unit 414. Balancing queue 
410 receives data packets, i.e. IP datagrams, from a plurality of participants over 
computer network 1 10. The data packets contain current opinion values and time 
stamps marking the relevant frames for the polling event. 

Line balancer 1 14 also includes balancing routing unit 414 that routes the 
data packets to an available polling server 1 1 8a- 1 1 8n. Thus, the data packets wait 
in balancing queue 4 1 0 until a suitable polling server 1 1 8a- 1 1 8n can be identified 
by balancing routing unit 414. Balancing routing unit 414 determines server 
availability by, for example, evaluating the capacity and current workload of each 



WO 01/53922 PCT/USO 1/02241 



-35- 



pollmg server 1 18a-l 1 8n, resources required by the submitted job, and other user- 
defined (i.e., poll/media manger 142) criteria. In one embodiment, the data packets 
are routed to the first available server. In another embodiment, the data packets can 
be routed to designated servers based on other criteria, such as, the type of poll 
5 (e.g., live or on-demand polling), geographic regions, and the like as would be 

apparent to one skilled in the relevant art(s). 

Therefore, load balancer 1 14 optimizes the use of computer resources by 
evenly distributing workloads to available polling servers 118a-118n within 
polling system 1 00. Moreover, load balancer 1 1 4 prevents polling system 1 00 from 

10 becoming overloaded. Load balancer 114, or a combination of multiple load 

balancer 1 14, supports the aggregation of an unlimited number of polling servers 
1 1 8a- 1 1 8n. Load balancer 1 14 is specialized hardware, and can be, for example, 
a BIG/ip® Controller developed by F5 Networks Inc., IOS server load balancing 
products from Cisco Systems, or similar hardware, software, or a combination 

15 thereof, as would be apparent to one skilled in the relevant art(s). 

C. Polling Servers 

FIG. 5 illustrates, according to one embodiment of the present invention, 
a block diagram of each polling server 118a-118n. FIG. 5 is a conceptual 
illustration of polling servers 118a-118n that allow an easy explanation of the 

20 present invention. That is, one or more of the blocks can be performed by the same 

piece of hardware or module of software. It should also be understood that 
embodiments of the present invention can be implemented in hardware, software, 
or a combination thereof. In such an embodiment, the various components and 
steps would be implemented in hardware and/or software to perform the functions 

25 of the present invention. 

Polling servers 118a-118n can be, for example, Intel®/NT computers, 
Intel® Linux boxes, or similar hardware, software, or a combination thereof, as 
would be apparent to one skilled in the relevant art(s). The total number of polling 
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servers 1 18a-l 18ncan be scaled to service an unlimited number of participants as 
would be apparent to one skilled in the relevant art(s). In one embodiment, each 
polling server 1 1 8a- 1 1 8n can support approximately 5,000 simultaneous sessions 
of live and on-demand streaming. Thus, polling system 1 00 can reach hundreds of 
5 thousands of participants daily. 

As shown in FIG. 5, each polling server 1 18a-l 18n in a representative 
embodiment, includes a memory unit 510 and work queue manager 514. Work 
queue manager 514 exchanges signals with load balancer 114 and collection and 
display server 122. Each polling server 1 18a-l 1 8n is assigned a unique network 

10 (e.g., IP) address that load balancer 1 14 uses to distribute the data packets across 

the individual polling servers 1 18a-l 18n, as discussed in reference to FIG. 4. 

In one embodiment, each polling server 1 18a-l 18n contains a duplicate 
copy of the entire database from user demographic database 130. In this 
embodiment, it would not matter which participant's response packets are routed 

15 to a given polling server 1 1 8a- 1 1 8n. Therefore, for large panel sizes, there should 

be no statistical differences among the results on each machine as the load 
balancing is uncorrelated to the data. 

In this embodiment, a copy of the database is sent to work queue manager 
514 from collection and display server 122, as discussed below. As shown in FIG. 

20 5, work queue manager 514 routes the database to memory unit 510 for storage. 

This configuration has been provided only by way of example and not limitation. 
Other configurations can be used without changing the scope of the invention. For 
instance, in another embodiment, a copy of the database can be routed direcdy to 
work queue manager 514 from user demographic database 130. In yet, another 

25 embodiment, a copy of the database can be routed directly to memory unit 510 

from either collection and display server 122 or user demographic database 1 30. 

FIG. 6 illustrates one embodiment of a memory resident master data table 
600 generated by work queue manager 514. Work queue manager 5 1 4 uses master 
data table 600 to manage and analyze the opinion value data located in the data 

30 packets. In one embodiment, work queue manager 5 1 4 is part of a multithreaded 
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Java application that manages the input of data from each participant through a 
unique socket. Therefore, in this embodiment, each participant represents a thread 
in a Java application on polling servers 1 18a-l 1 8n. Work queue manager 514 
examines the data packets to extract and add each new data items from the 
participants to master data table 600. In other words, each participant connection 
(socket) is polled by the Java application for data and the data is added to master 
data table 600. 

Referring to FIG. 6, the PollJD, UserJD and Timestamp fields identify 
where in master data table 600 each new opinion value data entry (shown in FIG. 
6 as the Value field) would be placed. Master data table 600 represents a sliding 
time window for each polling session and is organized by UserJD (i.e., 
participant's identification) and Timestamp (e.g., time or frame code). Work queue 
manager 514 inserts the Value data into the array element determined by the 
UserJD and Timestamp for the appropriate polling session (i.e., PollJD). The 
PollJD variable facilitates the support of multiple simultaneous polling sessions. 
Polling servers 118a-118n would maintain a memory resident window 
representing several minutes of polling data. This allows high latency data to 
"catch up" without slowing down system operations. 

D. Collection and Display Server 



FIG. 7 illustrates, according to one embodiment of the present invention, 
a block diagram of collection and display server 122. FIG. 7 is a conceptual 
illustration of collection and display server 122 that allows an easy explanation of 
the present invention. That is, one or more of the blocks can be performed by the 
same piece of hardware or module of software. It should also be understood that 
25 embodiments of the present invention can be implemented in hardware, software, 

or a combination thereof. In such an embodiment, the various components and 
steps would be implemented in hardware and/or software to perform the functions 
of the present invention. 
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As shown in FIG. 7, collection and display server ! 22 in a representative 
embodiment, includes a memory controller 710. checkpoint memory unit 714, 
demographic memory unit 71 8, polling databank 722, analyzing unit 726, graphics 
unit 730 and formatting unit 734. Memory controller 710 serves as a switch for 
5 routing signals among polling servers 1 18a-l 1 8n, checkpoint memory unit 714, 

demographic memory unit 7 1 8 and polling databank 722. Checkpoint memory unit 
714 communicates with data checkpoint and storage database 126. As discussed 
below, checkpoint memory unit 714 periodically routes signals (via memory 
controller 710) to polling servers 118a-l 18n and polling databank 722 to backup 

10 the data files. 

Memory controller 7 1 0 also routes signals from demographic memory unit 
7 1 8 to polling servers 1 1 8a-l 1 8n and polling databank 722. Demographic memory 
unit 7 1 8 communicates with user demographic database 1 30 to obtain a duplicate 
copy of the entire database from user demographic database 130. In one 

15 embodiment of the present invention, demographic memory unit 718 instructs 

memory controller 710 to route the duplicate copy of the database to polling 
servers 118a-l 18n and polling databank 722. In this embodiment, demographic 
memory unit 718 maintains coherency among resident memories within each 
polling server 1 1 8a- 1 18n and polling databank 722, as discussed below. 

20 Collection and display server 122 also includes polling databank 722. In 

one embodiment, polling databank 722 contains an aggregated master data table 
similar to the table 600 depicted in FIG. 6. Since each polling server 1 18a-l 18n 
contains a time randomized subset of opinion value data, polling databank 722 
instructs memory controller 710 to read each polling server 118a-118n, for 

25 example, every second to update the aggregated master data table. 

Analyzing unit 726 reads the aggregated master data table in polling 
databank 722 to generate a composite result representing the entire panel for each 
polling event. Analyzing unit 726 includes programmable instructions that 
provides for the easy addition and manipulation of data formats. Therefore, 

30 analyzing unit 726 is configured to produce one or more filtered views of the 
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composite results. In other words, the aggregated master data table contains a 
subset of user demographic data that can be used to provide database-filtered data 
views of the real time polling data. Applying filtering criteria, such as women in 
a specified age range, would cause the Include_Flag to be set to include only the 
value data meeting the filtered criteria. As such, in this example, the composite 
result would only include the opinion values in the designated filtered or 
demographic view, such as women in the specified age range. 

Collection and display server 122 also includes graphics unit 730 that 
creates graphical displays of the composite and/or filtered composite results. 
Graphics unit 730 collects the composite result and generates the appropriate 
graphical representation of the data for display. The graphical representation data 
is time stamped to support synchronization with the composite result. In one 
embodiment, graphics unit 730 creates data packets that separates the data layer 
(i.e., Value data) from the presentation layer (i.e., graphical representations) so that 
output formats and devices for the graphics data are independent from the formats 
and devices for the text data containing the composite results. 

For the graphics data, output formatted for PowerPoint® developed by 
Microsoft®, SAS® software developed by S AS Institute Inc., formats required by 
external video overlay generators, and the like, as would be apparent to one skilled 
in the relevant art(s), can be supported and independently maintained as needed. 
Graphics unit 730 provides for the independent management of data display 
formats. As such, PowerPoint® formats, and the like, or original graphical 
representations can be created by collection and display server 122 and managed 
by the viewer software on the client apparatuses. This can be accomplished by 
routing a Java code to the client apparatuses that includes programmable 
instructions for the addition of custom display applets. 

Polling system 100 can be configured to support both the server side and 
client side graphical display strategies. As discussed above, graphics unit 730 can 
embed instructions (e.g., Java code) into the data packets to control the display 
format at the client side. Alternatively, graphics unit 730 can embed formatting 
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instructions to allow the graphics data to be displayed according the client's 
format. In this instance, graphics unit 730 provides instructions, for example, to 
drive a video overlay creation device to support the insertion of a graphical overlay 
on top of the video signal from the server. 
5 Graphics unit 730 also supports multiple filtered views. Plotting multiple 

demographics on the same chart can be done in two ways: (1) use multiple data 
windows at, for example, 3 million bytes per table; and (2) run multiple passes on 
the aggregated master data table and output the demographics data sequentially. 
The first method can be faster since the data table is resident inside of graphics 

1 0 unit 730. However, the second method is generally more efficient because it would 

require less memory capacity. 

Collection and display server 122 also includes formatting unit 734. 
Formatting unit 734 receives the composite result data streams and graphical 
representation data streams from graphics unit 730. The data streams are 

15 synchronized and routed to media management server 134. Formatting unit 734 

can route only the composite result data, only the graphical representation data or 
both as instructed by polling management unit 3 14. 

The synchronized data stream output from formatting unit 734 also 
includes the time stamp from the originating client apparatuses that permits the 

20 opinion value to be synchronized to correspond with the media stream from which 

it was created. The data stream output with synchronization data is sent to the 
formatting unit 318 within media management server 1 34, where the data stream 
is correlated with the media stream for the matching event. 

Formatting unit 734 also receives several signals from formatting unit 318. 

25 Referring to FIGs. 3 and 7, polling management unit 314 can route signals (via 

formatting unit 318) to formatting unit 734 to set the data format and filtering 
criteria for analyzer unit 726; graphics formatting instructions for graphics unit 
730; routing instructions for formatting unit 734; and similar commands for 
controlling and managing the operations of collection and display server 122. 

30 Polling media/manager 142 can also use media management server 134 (and, 
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hence polling management unit 314) to centrally command and control the 
operations of polling servers 118a-118n and load balancer 114. In one 
embodiment, this can be implemented by routing a signal through formatting unit 
734 to memory controller 710 to be routed to polling servers 118a-118n (not 
shown). In another embodiment, each polling server 1 18a-l 18n, collection and 
display server 122 and load balancer 1 14 can have separate input devices and/or 
display GUIs (not shown) to permit poll/media manager 142 to control the 
respective operations. 

E. Data Checkpoint and Storage 



Referring again to FIG. Spoiling system 100 also includes data checkpoint 
and storage database 126. As discussed above, data checkpoint and storage 
database 126 controls the backup operations of the resident memories or 
transaction logs within the network components, such as, media management 
server 134, collection and display server 122 and each polling server 11 8a- 11 8n. 
15 A checkpoint command provides a point of synchronization between the data 

checkpoint and storage database 126 and other network components. For instance, 
referring to FIG. 5, work queue manager 514 contains a snapshot of current 
opinion values within its resident master data table 600. A copy of master data 
table 600 can also be retained in data checkpoint and storage database 126. Thus, 
20 if master data table 600 is damaged or lost in polling server 1 18a-l 18n, the data 

can be recovered by accessing the data checkpoint and storage database 126. 

Many methods can be used to perform a checkpoint For example, in one 
embodiment, data checkpoint and storage database 126 waits until all outstanding 
write requests to database 126 and the memories within the network components 
25 are complete. A checkpoint record is then written to data checkpoint and storage 

database 1 26. When the data needs to be recovered, only images for transactions 
processed after the checkpoint need to be applied. 
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Data checkpoints are written at a specified interval to facilitate error 
recovery conditions. In other words, data checkpoints can be written to data 
checkpoint and storage database 1 26 every N seconds, where the checkpoint time 
(N) is determined by the poll/media manager 142 and/or polling management unit 
3 1 4, as experience with the system 1 00 is accumulated. For example, a ten-minute 
memory resident data window for 5,000 users would require 3 million bytes (i.e., 
(5,000 users) x (bytes per Value) x (600 seconds), assuming one byte per value). 

F. User Demographic Database 

Referring to FIG. 1, polling system 100 also includes user demographic 
database 1 30 which contains a complete record of all participants' demographics. 
In one embodiment, user demographic database 130 is a relational database 
management system. As discussed above, user demographic database 130provides 
a duplicate copy of its records to media management server 134, collection and 
display server 122 and each polling server 1 1 8a-l 1 8n. 

As discussed in reference to FIG. 3, in one embodiment of the present 
invention, the database is generated and/or updated each time a participant 
registers for a polling event via registering unit 3 1 0. In another embodiment of the 
present invention, the database records can be generated or augmented by seeking 
demographic information from one or more third party proprietary databases (not 
shown). In this embodiment, user demographic database 130 can contact a third 
party proprietary database, such as Acxiom® developed by Acxiom Corporation 
or PRIZM™ developed by Claritas™ Inc. Accordingly, if the participant elects not 
to include certain demographics information, user demographic database 130 can 
obtain the information from a third party source. 

As discussed in reference to FIGs. 5 and 7, polling servers 1 1 8a-l 1 8n and 
collection and display server 122, each, contain a duplicate copy of the entire 
database from user demographic database 130. The database records are used by 
work queue manager 514 and analyzing unit 726 to provide valid demographic 
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interpretation of the polling data. Similarly, the database records are used to 
support demographically filtered views generated by graphics unit 730. 
Accordingly, referring to FIG. 6, a portion of user demographic database 130 is 
incorporated into master data table 600. Data streams representing composite 
results are created representing any and all of the recorded demographics. 

Referring to FIG. 7, an initial set of demographics is set by polling 
management unit 3 1 4 (as directed by poll/media manager 1 42) to determine which 
demographics would be processed in analyzing unit 726 and graphics unit 730. At 
any time during an polling event, poll/media manager 142 can cause the 
demographic display criteria to change. In one embodiment, the participant can. 
request demographic display changes by selecting an icon (not shown) disposed 
on client GUI 200. The appropriate display signal would be routed to polling 
management unit 3 14 for processing. 

To generate the composite results, filtered composite results and graphical 
representations, analyzing unit 726 and graphics unit 730 would interrogate the 
resident database information within the aggregated master data table. A 
participant's polling data that has no known demographic value would be excluded 
from the filtered views, unless poll/media manager 142 directs otherwise. 

User demographic database 1 30 supports both live and on-demand polling 
events. As such, analyzing unit 726 and graphics unit 730 have full access to 
updated user demographics to support post event data analysis, as well as real time 
analysis of live events. 

User demographic database 130 also provides for coherency among the 
duplicate database records distributed among the network components, namely 
polling servers 118a-118n and collection and display server 122. In one 
embodiment, polling management unit 314 suspends, at periodic intervals, all 
write and read requests to the database records in user demographic database 130 
and the network components. At this time, the demographics in the duplicate 
database records are synchronized with the records in user demographic database 
130. As a result, the network components can resume operations with the most 
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current demographics data for the participants. The periodic intervals for 
maintaining coherency can be adjusted by poll/media manager 142 at any time, as 
appropriate. 

IV. Software and Hardware Embodiments 

The present invention (e.g., system 100, media management server 134, 
collection and display server 122, polling servers 1 18a-l 18n, load balancer 1 14, 
user demographic database 1 30, data checkpoint and storage database 126, or any 
part thereof) can be implemented using hardware, software or a combination 
thereof and can be implemented in one or more computer systems or other 
processing systems. In fact, in one embodiment, the invention is directed toward 
one or more computer systems capable of carrying out the functionality described 
herein. 

Referring to FIG. 8, an example computer system 800 useful in 
implementing the present invention is shown. The computer system 800 includes 
one or more processors, such as processor 804. The processor 804 is connected to 
a communication infrastructure 806 (e.g., a communications bus, cross-over bar, 
or network). Various software embodiments are described in terms of this 
exemplary computer system. After reading this description, it will become 
apparent to a person skilled in the relevant art(s) how to implement the invention 
using other computer systems and/or computer architectures. 

Computer system 800 can include a display interface 802 that forwards 
graphics, text, and other data from the communication infrastructure 806 (or from 
a frame buffer not shown) for display on the display unit 830. 

Computer system 800 also includes a main memory 808, preferably 
random access memory (RAM), and can also include a secondary memory 810. 
The secondary memory 8 1 0 can include, for example, a hard disk drive 8 1 2 and/or 
a removable storage drive 814, representing a floppy disk drive, a magnetic tape 
drive, an optical disk drive, etc. The removable storage drive 814 reads from 
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and/or writes to a removable storage unit 8 1 8 in a well-known manner. Removable 
storage unit 818, represents a floppy disk, magnetic tape, optical disk, etc. which 
is read by and written to removable storage drive 8 14. As will be appreciated, the 
removable storage unit 818 includes a computer usable storage medium having 
stored therein computer software and/or data. 

In alternative embodiments, secondary memory 810 can include other 
similar means for allowing computer programs or other instructions to be loaded 
into computer system 800. Such means can include, for example, a removable 
storage unit 822 and an interface 820. Examples of such can include a program 
cartridge and cartridge interface (such as that found in video game devices), a 
removable memory chip (such as an EPROM, or PROM) and associated socket, 
and other removable storage units 822 and interfaces 820 which allow software 
and data to be transferred from the removable storage unit 822 to computer system 



800. 



Computer system 800 can also include a communications interface 824. 
Communications interface 824 allows software and data to be transferred between 
computer system 800 and external devices. Examples of communications interface 
824 can include a modem, a network interface (such as an Ethernet card), a 
communications port, a PCMCIA slot and card, etc. Software and data transferred 
via communications interface 824 are in the form of signals 828 which can be 
electronic, electromagnetic, optical or other signals capable of being received by 
communications interface 824. These signals 828 are provided to communications 
interface 824 via a communications path (i.e., channel) 826. This channel 826 
carries signals 828 and can be implemented using wire or cable, fiber optics, a 
phone line, a cellular phone link, an RF link and other communications channels. 

In this document, the terms "computer program medium" and "computer 
usable medium" are used to generally refer to media such as removable storage 
drive 814, a hard disk installed in hard disk drive 812, and signals 828. These 
computer program products are means for providing software to computer system 
800. The invention is directed to such computer program products. 
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Computer programs (also called computer control logic) are stored in main 
memory 808 and/or secondary memory 810. Computer programs can also be 
received via communications interface 824. Such computer programs, when 
executed, enable the computer system 800 to perform the features of the present 
5 invention as discussed herein. In particular, the computer programs, when 

executed, enable the processor 804 to perform the features of the present 
invention. Accordingly, such computer programs represent controllers of the 
computer system 800. 

In an embodiment where the invention is implemented using software, the 
10 software can be stored in a computer program product and loaded into computer 

system 800 using removable storage drive 814, hard drive 8 1 2 or communications 
interface 824. The control logic (software), when executed by the processor 804, 
causes the processor 804 to perform the functions of the invention as described 
herein. 

15 In another embodiment, the invention is implemented primarily in 

hardware using, for example, hardware components such as application specific 
integrated circuits (ASICs). Implementation of the hardware state machine so as 
to perform the functions described herein will be apparent to persons skilled in the 
relevant art(s). 

20 In yet another embodiment, the invention is implemented using a 

combination of both hardware and software. 



V, Exemplary Polling Session 

Referring to FIG. 9, flowchart 900 represents the general operational flow 
of an embodiment of the present invention. More specifically, flowchart 900 
25 shows an example of a control flow for collecting opinion research data over a 

network connection 110. 

Referring to FIG. 9, the control flow of flowchart 900 begins at step 901 
and pass immediately to step 910. At step 910, a participant uses a client 
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apparatus, such as, a computer, PDA, telephone, television, or the like, to register 
for a polling event This can be accomplished by clicking on an icon on the display 
GUI for the client apparatus. At step 914, a registering application program (i.e., 
Java applet) within the client apparatus would prompt the participant for a 
UserJD. If the participant has not previously registered with polling system 1 00, 
the participant would not have a UserJD and would be prompted by the 
registering applet to create a user profile at step 9 1 8. The user profile would collect 
demographic information about the participant, such as, age, gender, place of 
residence, political affiliations, ethnicity, marital status, and other demographics 
data as would be apparent to one skilled in the relevant art(s). 

Once the participant has created the user profile, the registering applet 
routes the profile to registering unit 310 as shown in FIG. 3. Registering unit 310 
sends the user profile to user demographic database 130 where a new record and 
UserJD is generated for the participant. At step 924, registering unit 314 
subsequently forwards the UserJD to the participant and informs polling 
management unit 314 that the participant's registration process has been 
completed. 

If the participant has previously registered with polling system 1 00 and has 
a UserJD, at step 922, the participant can simply enter the information into the 
client apparatus. The registering applet would subsequently route the UserJD to 
registering unit 310 where a copy of the participant's profile (i.e., demographic 
record) is collected from user demographic database 130. A copy of the user 
profile can be sent to participant so that the demographics data can be updated, as 
appropriate. After the participant has updated the user profile, or indicated that no 
change is required, at step 924, registering unit 310 would signal polling 
management unit 3 14 that the participant's registration has been completed. 

If the participant does not properly register with polling system 1 00, at step 
924, registering unit 310 notifies the participant that registration has been denied 
and the control passes to step 995. Otherwise, at step 926, polling management 
unit 314 receives a signal indicating that the participant has been properly 
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registered to take part in a polling session. Polling management unit 314 then 
instructs formatting unit 318 to transmit initialization instructions to the 
participant's client apparatus and set the operational modes for managing the 
polling session. 

The operational modes include the synchronization, transmission, resetting 
and time-out schemes. The synchronization scheme determines how the media 
streams from formatting unit 318 and opinion value data streams from the client 
apparatus would be time stamped (e.g., time or frame code). The transmission rate 
determines the frequency for reporting opinion value data streams from the client 
apparatus and routing composite results and graphical representations from 
collection and display server 122. For reporting the data streams from the client 
apparatus, the transmission rate can be a periodically scheduled rate or the 
reporting can be event-driven. The resetting scheme determines whether polling 
indicator 218 is reset to a neutral position after each input. For example, if no 
response is entered for a designated time, polling indicator 218 can be 
automatically returned to a neutral position, e.g., the "50%" mark shown in FIG. 
2. The time-out scheme determines whether the polling session would be 
terminated or temporarily suspended after a specified period of time has elapsed. 
These instructions can be routed to a Java applet within the client apparatus which, 
in turn, manages the operational modes for the polling session. 

In one embodiment of the present invention, polling management unit 3 1 4 
maintains a list of registered participants that have been approved to partake in a 
polling session. Polling management unit 314 periodically provides the 
authorization list to formatting unit 318 that, in turn, sends initialization and/or 
management instructions to the client apparatuses of the authorized participants. 
The periodic rate can be adjusted by poll/media manager 142. Accordingly, at any 
given time during a polling session, polling system 100 can initialize, terminate or 
modify the operational modes for the participant's polling session. 

After the polling initialization and management instructions have been 
routed, at step 930, polling management unit 314 instructs formatting unit 3 1 8 to 
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start sending the media streams of the event to the client apparatus. The media 
streams include time stamps (e.g., time or frame codes). By using a single time 
line, all media on a page can be properly time coordinated and synchronized. 
Formatting unit 318 embeds formatting instructions into the media streams to 
make them compatible with the client apparatus. A markup language, such as 
HTML or SMIL, can be used to coordinate the media streams for display on the 
client apparatuses. 

At step 934, the participant interacts with client apparatus to indicate an 
opinion value about the media streams. Referring to FIG. 2, the participant 
indicates an opinion value by interacting with an input device (not shown) that 
moves polling indicator 218 to designate a discrete value between 0 and 100. A 
polling API interfaces with GUI 200 and the input device to provide real time 
visual feedback by moving polling indicator 218. 

At step 938, the client apparatus synchronizes (e.g., time stamps) and 
routes a data stream containing a current opinion value and time stamp to load 
balancer 114 at the transmission rate set by polling management unit 314. Thus, 
each time the participant indicates a current opinion, the polling API establishes 
a quantitative value for the opinion depicted by polling indicator 218. A Java 
applet takes the current opinion value, synchronizes the opinion value with the 
media streams and routes the synchronized opinion data to load balancer 1 14. 

At step 942, load balancer 1 14 collects the data stream consisting of a 
current opinion value and time stamp and routes the data stream to an available 
polling server 1 18a-l 18n. Referring to FIG. 4, the data stream generally waits in 
balancing queue 410 until a suitable polling server 1 18a-l 18n can be identified by 
balancing routing unit 414. Balancing routing unit 414 determines server 
availability by, for example, evaluating the capacity and current workload of each 
polling server 1 1 8a- 1 1 8n, resources required by the submitted job, and other user- 
defined (e.g., poll/media manger 142) criteria. 

At step 946, the recipient polling server 118a-118n analyzes the data 
stream to tabulate the participant's opinion. The recipient polling server 1 1 8a-l 1 8n 
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has a duplicate copy of the entire database from user demographic database 130. 
Referring to FIG. 5, a copy of the database is sent to work queue manager 514 
which generates a memory resident master data table, shown in FIG. 6 as master 
date table 600. Work queue manager 5 1 4 uses master data table 600 to manage and 
analyze the opinion value data located in the data stream. The participant 
represents a thread in a Java application on polling servers 1 18a- 1 1 8n, and the 
participant's connection (socket) is polled by the Java application for data which 
is added to master data table 600. Work queue manager 514 inserts the Value data 
into the array element determined by the User_ID and Timestamp fields for the 
appropriate polling session, PolHD. 

At step 950, collection and display server 122 reads the recipient polling 
server 1 18a-l 18n at a periodic rate, for example every second, and generates a 
time aggregated master data table. Referring to FIG. 7, analyzing unit 726 utilizes 
the aggregated master data table to create a composite result representing an entire 
panel of participants. Since the aggregated master data table also includes a 
duplicate of the database from user demographic database 130, analyzing unit 726 
can also utilize the aggregated master data table to provide database-filtered data 
views of the real time polling data. Applying filtering criteria, such as gender or 
age, analyzing unit 726 can create composite results for each demographic or 
filtered view. 

In one embodiment of the present invention, collection and display server 
1 22 can generate a filtered view by making a selected subset of user data memory 
resident and associating the user data with the UserJDs for all participants. 
Referring to FIG. 7, analyzing unit 726 utilizes the Include_Flag, associated with 
each UseMD, in the aggregated master data table to identify the Value data to be 
assembled into the filtered or demographic view. The selected Include_FIag is 
turned on if the user demographic data matches the filtering criteria. For example, 
by setting the Include_Flag(s) to select all females between the ages of 18-32, the 
filtered view would only include the composite results of participants in that 
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particular demographic group. If analyzing unit 726 sets all the IncludeFlags to 
"ON," the composite result would include all participants' responses. 

Additionally, at step 950, collection and display server 122 creates 
graphical displays of the data. Referring to FIG. 7, graphics unit 730 creates the 
appropriate graphical representation of the data to be displayed on the client 
apparatus. Depending on the filtering criteria, graphics unit 730 can generate 
graphical representations for each filtered view. Formatting unit 734 receives a 
data stream containing the composite result(s) and graphical representation(s) of 
the panel's opinion. The data stream also includes time stamps that synchronizes 
the data to correspond to the media streams where the opinions were created. 
Formatting unit 734 routes the data stream to media management server 134. 

At step 954, media management server 134 receives the data stream 
containing the composite result(s) and graphical representation(s). Referring to 
FIG. 3, formatting unit 318 receives the data stream and determines which 
registered participants are approved to receiving polling data. As discussed in 
reference to step 926, polling management unit 3 14 periodically transmits signals 
to formatting unit 318 to identify all authorized participants for each polling 
session. 

If the participant has been authorized to receive, or continue to receive, 
polling data, at step 958, formatting unit 3 1 8 formats and sends the data stream to 
media routing unit 322. Media routing unit 322 sends the data stream to the 
participant's client apparatus, along with the media streams of the polling event. 
The control flow then passes to step 934 where the participant continues to interact 
with the media streams to indicate an opinion value. 

However, if, at step 954, the participant's polling session has been 
terminated, either by the participant or polling management unit 314, the 
participant would not be identified as an authorized participant. In this instance, 
at step 962, formatting unit 318 and media routing unit 322 would not send 
additional media streams to participant. If instructed by polling management unit 
3 1 4, formatting unit 3 1 8 can send a final data stream showing the composite result 
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to the participant. This can occur if the event has ended and the panel of 
participants have been authorized to receive the final opinion value. Upon 
termination of the participant's polling session, the control flow ends as indicated 
by step 995. 



5 VI. Opinion Research Navigation 

In one embodiment of the present invention, a navigation guide is provided 
to assist, direct or lead a respondent or other system user during a session with 
media management server 134 or other components of real time polling system 
100. The navigation guide can be provided as text, images or video streams. 

10 In one embodiment, the navigation guide includes a video moderator that 

prompts the user during the session. FIG. 15 illustrates one embodiment of GUI 
1500 containing a navigation screen 1504 the displays a video moderator. For 
example, when a respondent registers to initiate a polling session, as discussed 
above, the video moderator would appear in navigation screen 1504 to assist the 

15 respondent during the registration process. The video moderator can ask the 

respondent to select a prize or incentive for participating in the polling session. 
The prize can be money, flooz, movie tickets, coupons, and the like. The video 
moderator could also be activated to provide data if the respondent activates a help 
icon or request similar data from a context menu. 

20 As discussed, GUI 1500 is one embodiment of a navigation guide. FIG. 

16 illustrates another embodiment of a GUI (shown as GUI 1600) containing a 
navigation screen 1 504. In other embodiments, navigation screen 1 504 can present 
only text, audio, graphics and like media and multimedia. 

VIL Poll Creation and Management 
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In one embodiment of the present invention, Interview Script Markup 
Language (ISML) is used to prepare a polling event, including surveys and 
interviews. ISML is an interpretative opinion research authoring language that 
utilizes XML standards and provides multimedia control. ISML can be used to 
5 cr ^ te ^ dm ^ a geinteractiveopinionresearchoveracomputernetwork,including 
the Internet. ISML can manage video with textual questions simultaneously and 
seamlessly. 

FIG. 17 illustrates a GUI 1700 of an authoring environment that can be 
used to debug the programming code generated in ISML. Window 1702 can be 
10 used to edit the code or watch the code play. Window 1 702 can also be used to 

edit timing, text and video controls, including multiple video clips and entry point 
into video clips, and the like. Window 1702 can also be used for creation of 
questions, assignment of data variables to answers, instructional messages, 
interaction without presentation software such as Microsoft PowerPoint®, and 
15 supporting cutting and pasting into the editing environment. 

ISML is meant to fully define how an interview is experienced by the user 
(e.g., respondent) as well as capture the data from that session and provide it in a 
form that can be further processed by other database tools, such as SAS, Oracle®, 
Microsoft Access®, and the like. 

The language is a simple procedural listing of events and responses in an 
HTML-like ASCII format. The script can be readily created and viewed in a text 
editor, but an interactive graphical script-maker is contemplated as part of the 
present invention 

There are four types of events that can be created: Script control, the 
25 playing of media, asking of a questions, and presentation of static information. The 

language is extensible, and more types can be added as required. 

The script is interpreted sequentially like most procedural languages, 
executing the events in order they appear in the script. Most event tags are single 
line, but the <PLAY> and <ASK> tags have corresponding </PLAY> and 
30 </ASK> tags that define their scope, or range of influence. 



20 
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All events (with the exception of <PLAY>) can be nested within another, 
allowing for branching of questions and media based on responses of previous 
questions. 

ISML is case-sensitive, e.g., uppercase for tag commands and lowercase 
5 for parameter tags. Obviously, the parameters themselves can be either case, as 

desired. 

Document Format Notation: 

Event names are BOLD CAPITALIZED. Parameter names are 
lowercase. Parameter values are in Italics. Optional parameters are 
10 in brackets Q. 

Script Control Events 

When the script is completed, the DONE tag will cede control of the script 
to a page defined by 'newpageurl'. The TAG tag defines a place in the 
script to jump to with the GOTO tag. The URL tag will stop the script and 
15 go to 'newpageurl' 

<DONE=newpageurl> 
<GOTO=tagname> 
<TAG=tagname> 
<DONE=newpageurl> 

20 The Playing of Media 

Adding the PLAY tag into the script causes the Media Player to play 
particular media stream, (defined by the 'clip 5 parameter) audio or video, 
from some position within the stream to some other time (defined by the 
'start* and 'end' parameters). 
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<PLAY=cIipname start=start end=end > 
<VPLAY> 



To have events occur at a particular time while media is playing, use the 
AT tag within the play tag's scope (defined as being within the ending 
</PLAY>tag). 

<AT=time> 

When an AT tag is encountered, the interpreting of the script is stopped 
until the Media Player reaches that time, and then the events following the 
AT tag will be executed until the <#>LAY> tag is reached, when playing 
continues until the end of the media is reached. 

There are two other media tags. The PAUSE tag will cause the Media 
Player to stop playing, and the RESUME tag will cause it to resume 
playing where it left off. The STOP tag will fast forward to the end of the 
clip. The CUE tag will cue the media to a point specified by 'time'. 

<RESUME> 
<PAUSE> 
<STOP> 
<CUE=time> 

Moment by Moment Polling 

While the media is playing, moment-by-moment information can 
be captured to the destination defined by 'data' identifying name. 
This command must be within a <PLAY> </PLAY> tag set. To 
start polling at a particular time, place it after an <AT=> tag 
specifying where you want to start. 
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Data will be collected from this point until the script reaches a 
</POLL> tag, placed after the </PLAY> tag to capture data until 
the end of the media being player, or at some earlier point in the 
script after an <AT=> tag specifying where you want to end the 
5 data collection. 

<POLL=data=data set name> 
</POLL> 



Example: 

<PLAY=reagan.asf start=0 end=30> 
10 <AT=5> 

<POLL=polll> 
</PLAY> 
</POLL> 

<DONE=nextpage.html> 

1 5 General Text formatting Options 

Any single tag must appear on a single line and not contain any line 
breaks or carriage returns within in it. To add a break, use the line 
break option described below. When specifying the text for a 
<SAY>, <ASK> or <ANS> tag there are a number of text 
20 formatting options that can change the way text is displayed. 

Line Break 

Lines can be broken up by inserting a ' | ' (a vertical bar) in 
the text. All text following the 4 | ' will appear on the 
following line. 



BNSDOCID: <WO 0153922A2J_> 



WO 01/53922 



PCT/US01/02241 



-57- 

Center Line 

The line can be centered on the screen by prefacing the line 
with a 4 ~ ' (tilde). Please note that each time you break a 
line using the line break 4 | * or the line wraps beyond the 
edge, the centering will revert to left justified. The * ~ ' 
should appear after any leading * * marks. 

Highlight 1st Letter 

The first letter of a line can be highlighted with a red circle 
by prefacing the line with a 4 A 5 (caret). The 4 A 5 should 
appear after any leading * | ' marks. 

Add a Bullet 

Adding an ' @ ' (ampersand) to the start of any line will 
cause a circular bullet to precede the answer, and the text 
will be indented accordingly. The ' @ * should appear after 
any leading ' | ' marks. 

Indent 

Adding a 4 1 ' (left accent) to the start of any line will cause 
a circular bullet to precede the answer, and the text will be 
indented accordingly. The 4 * * should appear after any 
leading 4 1 ' marks. You may use multiple indents to shift 
the line further to the right. 

Embed an Image 

You can add images within a line of text by adding the 
following tag to the text line: IMG::filename, where 
'filename' can be any GIF or JPG file you want to draw on 
that line. Make sure the IMG is uppercase and there is no 
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space between it and the filename. Also make sure to add 
the proper file extension (.jpg or gif) and there is a space 
after that and the next text on the line. 

For Example: 

5 <S AY=Powered by IMG: :ismlogo.gif by Ntercept> 

The Presentation of Static Information 

The SAY tag will cause text to be displayed on the screen, but not 
expect any interaction from the respondent. 

<SAY=text to display> 

10 Example: 

<PLAY=reagan.asf start=0 end=30> 
<AT=0> 

<SAY=This is the beginning of the show> 
<AT=5> 

15 <SAY=This is the middle of the show|New line> 

</PLAY> 
, <S A Y=This is the end of the show> 
<DONE=nextpage.html> 



Showing Graphic Images 

20 The IMG tag will cause a bitmap image to be displayed on the 

screen, but not expect any interaction from the respondent. 
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The 'justify tag will allow you to change the horizontal placement 
options on the screen. The default is left justified, adding 
4 justify=center' makes it centered and 'justify=right' makes it right 
justified. 

The 'align' tag determines whether the image will be aligned from 
the top or the bottom. The default causes the image to be aligned 
with the bottom of the current position. Adding an 'align=top' will 
cause the image to be drawn anchored from the top. 

<IMG=filename [justify=side] [align=endt]> 

<PLAY=reagan.asf start=0 end=30> 
<AT=0> 

<IMG=reagan.jpg justify=center> 
<AT=5> 

<IMG=reagan2.gif> 
</PLAY> 

<S AY=This is the end of the show> 
<DONE=nextpage.html> 



10 



15 



The Asking of Questions 

The ASK tag will automatically pause any media that is playing 
and present some text on the screen, and ask the respondent to 
react, using either the dial poll mechanism or through direct 
keyboard hits. There are three kinds of questions:, Rating, Yes/No, 
and Multiple choice 
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The question text will be displayed on the screen. Later versions of 
the interpreter will allow formatting and image commands to be 
embedded and shown. The 'data' parameter defines the data set 
name the response will be stored under. The 'style' parameter 
5 defines any special asking modes, such as multiple select (multi) 

and verbatim response (verb). 

<ASK=question text data=data set name [style=stylecode]> 
</ASK> 

All answers are defined by adding ANS tags within the scope of 
10 the ASK event (defined as being within the ending </ASK> tag). 

Playing is resumed unless otherwise stopped within the answer. If 
you want to direct where the script goes upon a particular answer, 
add a 'goto=* tag along with the tag name you want the script to 
execute from. 

1 5 <ANS answer text [goto=tag]> 



Rating Questions: 

Since rating questions have no preset answers, no ANS tags are 
required. When the script encounters the ASK tag, the respondent 
will be prompted to respond to the displayed question by setting 
20 the dial poll dial to the desired value and clicking. 

Example: 

<ASK=Rate Reagan as a president? data=questionl> 
</ASK> 
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<DONE> 
Verbatim Questions: 

Since verbatim questions have no preset answers, no ANS tags are 
required. When the script encounters the ASK tag, the respondent 
5 will be prompted to respond to the displayed question by typing 

their response in the text box below the question. 

Example: 

<ASK=What do you say? data=questionl verb=true> 
</ASK> 

10 <DONE> 



Yes/No Questions: 

Yes/No questions have two answers, so 2 ANS tags are required. 
When the script encounters the ASK tag, the respondent will be 
prompted to respond to the displayed question by setting the dial 
poll dial. Underneath the question, the 2 possible answers are 
displayed 

The dial no longer displays a range of 0-100, but only Yes or No, 
depending on which side it is turned to. In addition, the answer text 
will be highlighted to match the dial's current choice. 

Adding a '@' will cause a circular bullet to precede the answer, 
and the text will be indented. 'No' MUST proceed 'Yes' in the 
listing. 
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Example: 

<ASK=Was Reagan an good president 

<ANS=@ No, not so good goto=quit> 
<ANS=@ Yes, he was Great goto=stay> 

5 </ASK> 

<DONE=nextpage.html> 



Multiple Choice/Selection Questions: 

Multiple choice questions have any number of possible answers, so 
that many ANS tags will be required. When the script encounters 
10 the ASK tag, the respondent will be prompted to respond to the 

displayed question by setting the dial poll dial. Underneath the 
question, the possible answers are displayed 

The dial no longer displays a range of 0-100, but 0 to the number 
of answers, depending on its position. In addition, the answer text 
15 will be highlighted to match the dial's current choice. 

If the optional 'style=verb' parameter is added to the ASK tag, 
multiple responses will be able to be selected. Make sure a 'goto^ 
parameter is added to the ANS tag you wish to exit on. 



Adding a *@' will cause a circular bullet to precede the answer, 
20 and the text will be indented. 

. Example: 
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<ASK=How old are you? data=question3> 

<ANS=@ Under 18> 

<ANS=@ 18to25> 

<ANS=@ 26 to 40> 

<ANS=@ 41 to 65 goto=old> 

<ANS"Over 65> 
</ASK> 

<DONE=nextpage.html> 



A More Complex ISML Example 

<ASK=Are you registered to vote? data=question2> 
<ANS=Yes> 
<ANS=No> 
<GOTO=quit> 
</ASK> 

<ASK=What is your political affiliation? data=question2> 
<ANS=Democrat goto=dem> 
<ANS=Republican goto=rep> 
<ANS=Other> 
</ASK> 
<GOTO=quit> 
<TAG=dem> 

<PLAY=carter.asf start=0 end=60> 
<AT=20> 

<ASK=Did you vote for him? data=question8> 
<ANS=Yes> 
<ANS=No> 
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<STOP> 
</ASK> 

<ASK=Was he an good president? data=question3> 
<ANS=Yes> 

5 <ANS=No> 

</ASK> 
<POLL=polll> 
</PLAY> 
</POLL> 
10 <GOTO=quit> 
<TAG=rep> 

<PLAY=reagan.asf start=0 end=60> 
<AT=25> 

<ASK=Was he an good president? data=question3> 
15 <ANS=Yes> 

<RESUME> 
<ANS=No> 
<STOP> 
</ASK> 

20 <POLL=poll2> 

<AT=45> 
</POLL> 
</PLAY> 
<TAG=quit> 
25 <DONE=nextpage.html> 



TAG COMMANDS 

<DONE=newpageurl> 
<GOTO=tagname> 
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<TAG=tagname> 

<PLA Y=clipname start=starttime end=endtime> 

</PLAY> 

<AT=time> 

<POLL=data set name> 

</POLL> 

<RESUME> 

<PAUSE> 

<STOP> 

<CUE=time> 

<S AY=text to display> 

<ASK=question text data=data set name [style=verb]> 
</ASK> 

<ANS answer text [goto=tagname]> 



DATA STORAGE 

Aside from the script, which defines the respondent's experience, 
any tool supporting ISM must have a method for communicating 
the results of the respondent's session to a database for analysis. 
This can be a simple as a large ASCII file that describes to results 
or a direct interface to a database residing in an Oracle-based 
server. 

The constituent parts are the same regardless of how they are 
stored. For each data collection opportunity (a multiple-choice 
question, moment-by moment poll, etc) There needs to be an ASK 
and ANS structure set sent to define the framework where a series 
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of corresponding RES structure responses to this opportunity will 
reside. 

Even though the types of questions may look disparate when asked 
during the interview, and may be interpreted differently in later 
analysis, from a data collection standpoint, they are basically stored 
the same. 



ASK Template Structure 

questionType: Defines the style of question (Y/N, poll, etc). 

questionText: Text of question asked during question . 

pollID: What poll number was this asked in. 

startTime: What time question was asked. 

askID: Identifier for the question! 



ANS Template Structure 



answerNumber: 

answerText: 

pollID: 

startTime: 

askID: 



Number of answer 0 to number of answers- 1 
Text of answer asked during question . 
What poll number was this asked in. 
What time question was answered. 
Identifier that says what question this is in 
response to. 



RES Response Structure 

pollID: What poll number was this asked in. 

responseText: Text of response asked during question . 
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respondentlD: Respondent ID. 

timeStamp: Time associated with response (0 for non- 

poll events) 

value: Value associated with response. 

askID: Identifier that says what question this is in 
response to. 

questionType codes 

Yes/No type=101 

Multiple-Choice type= 1 02 

Multiple-Selection type= 1 03 

Rating type=104 

Poll type=105 

Verbatim type=106 



Examples 

Yes-No / Multiple-Choice / Multiple-Selection Question 

For Yes/No, type=l 01 

For Multiple-Choice, type=102 

For Multiple-Selection, type=l 03 

Once, on poll setup: 

ASK: 

questionType=type 
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questionText="Do you Agree? 

pollID=1002 

askID=questionl 

ANS: 



questionText="Yes" 

pollID=1002 

askID=questionl 

For each response by the respondent to this question: 



5 



answerNumber=0 



questionle^'TNlo' 

pollID=1002 

askID=questionl 



ANS: 



10 



ans werNumber= 1 



15 



RES: 



responseText= 1 
polIID=1002 



20 



repondentID=l 345345 
askID=questionl 

value=NumberOfQuestionSelected 
timeStamp=0 
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Multiple selection value responses are bit-map encoded. Rather 
than store the actual number, the bit position of the 32-bit value 
represents which items were selected. 

for example: 

1 means the 1st is chosen 

2 means 2nd is chosen 

3 means both 1st and 2nd are chosen 

4 means 3rd is chosen 

5 means 1st and 3rd are chosen 
etc... 



Rating Question 

Once, on poll setup: 
ASK: 

questionType= 1 04 
questionText="Rate him 0-100" 
pollID=1002 
askID=ratingl 

For each response by the respondent to this question: 
RES: 
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responseText= ,m 
pollID=1002 
repondentID=l 345345 
askID=ratingl 

value=returnedNumberFromDial 
timeStamp=0 

Moment-by-Moment Polling 

Once, on poll setup: 
ASK: 

questionTy pe= 1 05 

questionText="Rate him over time" 

pollID=1002 

startTime=164 

askID=polll 

For each time returned by respondent: 

RES: 

responseText="" 
pollID-1002 
repondentID=1345345 
askID=polll 

value=returnedNumberFromDial 
timeStamp=timeOfSample 
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Verbatim Question 

Once, on poll setup: 
ASK: 

questionType=106 
questionText="Any suggestions?" 
pollID=1002 
askID=verbatiml 

For each time returned by respondent: 
RES: 

responseText="Make the process shorter." 

pollID=1002 

repondentID=1345345 

askID=polll 

value=0 

JAVA Interface 

The ISM delivery mechanism sends the database the data 
via a Java interface as follows, where arrays of various 
types of data packets are sent to and from the database. The 
arrays can be one or more elements, making the amount 
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and frequency the responsibility of the delivery mechanism, 
rather than the database. 

boolean putASK(ASKPacket[]); 
ASKPacket[]=getASKO; 

boolean putANS(ANSPacketQ); 
ANSPacketQ=getANSO; 

boolean putRES(RESPacketQ); 
RESPacket[]=getRESO; 

public class ASKPacket implements java.io.Serializable 
{ 

int mType; 
int mPollID; 
int mAskID; 
int mStartTime; 
String mAskText; 



public class ANSPacket implements java.io.Serializable 
{ 

int mType; 
int mPollID; 
int mAskID; 
int mStartTime; 
String mAnsText; 
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public class RESPacket implements java.io.Serializable 
{ 

int mPollID; 
int mAskID; 

5 int mRespondentID ; 

int mTimeStamp; 

int mValue; 
String mResText; 
} 

10 

NOTE: int's are 32-bit signed integers 



VIIL Conclusion 

While various embodiments of the present invention have been described 
above, it should be understood that they have been presented by way of example, 
15 and not limitation. It will be apparent to persons skilled in the relevant art(s) that 

various changes in form and detail can be made therein without departing from the 
spirit and scope of the invention. Thus, the present invention should not be limited 
by any of the above described exemplary embodiments, but should be defined only 
in accordance with the following claims and their equivalents. 
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What Is Claimed Is: 

1. A graphical user interface for interactively presenting opinion 
research data on a display coupled to a data processing device, said 
data processing device configurable to exchange opinion research 
data with a data processing server in real time or on a periodic 
basis, comprising: 

an indicator for generating a graphical metaphor of an 
opinion value from a user, said user operating an input device in 
communication with the display to create a designated opinion 
value regarding an event; and 

an indicator scale for showing an array of possible opinion 
values for said user to select said designated opinion value. 

2. An interface of claim 1, wherein said indicator is a slider, said 
slider including an area disposed thereon for dynamically 
displaying a discrete value of said designated opinion value. 

3. An interface of claim 1, wherein said indicator is a slider, said 
slider configurable to traverse in a direction selected from a group 
consisting of vertical, horizontal* arc, circular, and three- 
dimensionally. 

4. An interface of claim 1 , wherein said indicator is a slider and said 
indicator scale includes an area located at opposing ends of said 
indicator scale to dynamically display a graphical metaphor 
selected from a group consisting of facial expressions, hand 
signals, directional signals, arrows, and plus/minus signs. 

5. An interface of claim 1, wherein said indicator scale includes 
textual labeling along the axis of said indicator scale, said textual 
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5 labeling dynamically changing to specify a distinct opinion value 

from said array of possible opinion values as said indicator is 
moved along said axis, said axis being straight or curved. 

6. An interface of claim 1 , further comprising a section for displaying 
one or more graphical representations related to said designated 
10 opinion value. 



15 



25 



An interface of claim 6, wherein said graphical representations 
display said designated opinion value as a function of time, said 
graphical representations including at least one item selected from 
a group consisting of graphs, bar charts, pie charts and histograms. 



8. An interface of claim 7, further comprising at least two of said 
graphical representations, a first of at least two of said graphical 
representations depicting an opinion value for said user and a 
second of at least two of said graphical representations depicting 
a composite opinion value for a panel of users providing opinion 

20 data for said event. 

9. An interface of claim 8, wherein said first of at least two of said 
graphical representations is disposed adjacent to said second of at 
least two of said graphical representations. 



10. An interface of claim 8, wherein an overlay of said first of at least 
two of said graphical representations is disposed on said second of 
at least two of said graphical representations. 

11. An interface of claim 8, wherein said first of at least two of said 
graphical representations is displayed in a first color and said 
second of at least two of said graphical representations is displayed 
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in a second color, said second color being readily distinguishable 
from said first color. 

An interface of claim 8, wherein said first of at least two of said 
graphical representations is displayed in a first line style and said 
second of at least two of said graphical representations is displayed 
in a second line style, said second line style being readily 
distinguishable from said first line style. 

An interface of claim 7, further comprising an event window for 
displaying video, audio or text, each related to said event. 

An interface of claim 13, wherein a time axis for said graphical 
representations includes time marks associated with said 
designated opinion value, said time marks providing an index into 
said event, wherein activation of one of said time marks by said 
user operating said input device produces a dynamic or static 
display of said video or said text, said video or said text 
corresponding to said designated opinion value associated with 
said activated time mark. 

An interface of claim 14, wherein said graphical representations 
include one or more speaker icons associated with one or more 
speakers engaging in said event, wherein activation of each speaker 
icon highlights said temporal axis associated with said associated 
speaker, whereby activation of one of said time marks produces a 
dynamic or static display of said video or text corresponding to 
said designated opinion value associated with said activated time 
mark for said associated speaker. 
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An interface of claim 14, wherein said event window includes 
video and text, each related to said event, said text including 
captions from said video. 

An interface of claim 14, wherein said time axis includes a slider, 
wherein said user operates an input device to move said slider 
along said time axis to activate said time marks. 

An interface of claim 14, wherein said graphical representations 
include one or more speaker icons associated with one or more 
speakers engaging in said event, each speaker icon including a 
slider line, wherein said user operates an input device to move a 
slider along said slider line to activate one of said time marks 
associated with said associated speaker, whereby activation of said 
time mark produces a dynamic or static display of said video or 
said text corresponding to said designated opinion value associated 
with said activated time mark for said associated speaker. 

An interface of claim 13, further comprising a script window, 
wherein said event window includes video or audio related to said 
event and said script window includes text synchronized with said 
video or said audio. 

An interface of claim 1, wherein said input device is a touch 
screen, wherein said user touches said indicator to create said 
designated opinion value. 

An interface of claim 1, wherein said input device is a voice 
recognition unit, wherein said user provides voice commands to 
instruct said indicator to create said designated opinion value. 
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An interface of claim 1, wherein said input device is a biometric 
device disposed on a body part of said user, wherein measurements 
from said body part instructs said indicator to create said 
designated opinion value. 

An interface of claim 22, wherein said body part is an hand, 
wherein said user moves said hand in three-dimensional space to 
instruct said indicator to create said designated opinion value. 

A graphical user interface for interactively presenting opinion 
research data on a display coupled to a data processing device, said 
data processing device configurable to exchange opinion research 
data with a data processing server in real time or on a periodic 
basis, comprising: 

a section for displaying one or more graphical 
representations related to a designated opinion value produced on 
the data processing device in regards to an event; and 

an event window for displaying video, audio or text, each 
related to said event. 

An interface of claim 24, wherein said graphical representations 
display said designated opinion value as a function of time, said 
graphical representations including at least one item selected from 
a group consisting of graphs, bar charts, pie charts and histograms. 

An interface of claim 25, wherein a time axis for said graphical 
representations includes time marks associated with said 
designated opinion value, said time marks providing an index into 
said event, wherein activation of one of said time marks by said 
user operating said input device produces a dynamic or static 
display of said video or said text, said video or said text 



PCT/US01/02241 



-79- 

corresponding to said designated opinion value associated with 
said activated time mark. 

An interface of claim 26, wherein said graphical representations 
include one or more speaker icons associated with one or more 
speakers engaging in said event, wherein activation of said speaker 
icons activates said time marks associated with said associated 
speakers, thereby producing a dynamic or static display of said 
video or said text corresponding to said designated opinion value 
associated with said activated time marks. 

An interface of claim 24, further comprising a script window, 
wherein said event window includes video or audio related to said 
event and said script window includes text synchronized with said 
video or said audio. 

A method for displaying opinion data collected from a participant 
on a display coupled to a data processing device, said participant 
operating an input device in communication with said display to 
create a designated opinion value in response to an event, said data 
processing device configurable to exchange said opinion data with 
a data processing server in real time or on a periodic basis, 
comprising the steps of: 

producing on said display an indicator for generating a 
graphical metaphor of an opinion value from said participant; and 

producing on said display an indicator scale for showing an 
array of possible opinion values, wherein said participant operates 
the input device to manipulate said indicator to select said 
designated opinion value from said array of possible opinion values 
in response to the event. 
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5 30. A method of claim 29, further comprising the step of dynamically 

displaying on said indicator a discrete value for said designated 
opinion value in response to said participant manipulating said 
indicator. 

31. A method of claim 29, further comprising the step of displaying a 
10 textual labeling along an axis of said indicator scale, wherein said 

textual labeling dynamically changes to specify a distinct opinion 
value from said array of possible opinion values as said indicator 
is moved along said axis. 

32. A method of claim 29, further comprising the step of displaying 
1 5 one or more graphical representations for said designated opinion 

value. 

33. A method of claim 32,wherein said graphical representations are 
selected from a group consisting of a graph, chart, and histogram. 

34. A method of claim 32, wherein said graphical representations are 
20 generated for display on the data processing device. 

35. A method of claim 32, wherein said graphical representations are 
generated on the data processing server that analyzes said 
designated opinion value, said graphical representations being 
generated and sent to the data processing device for display. 

25 36. A method of claim 32, wherein said graphical representations are 

generated on a data processing server that analyzes said designated 
opinion value, said graphical representations being generated and 
sent on demand to the data processing device for display. 
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A method of claim 29, further comprising the step of displaying 
one or more graphical representations of a plurality of designated 
opinion values as a function of time. 

A method of claim 37, wherein said step of displaying graphical 
representations comprises the step of displaying at least two 
graphical representations, wherein a first of at least two of said 
graphical representations depicts an opinion value for said 
participant and a second of at least two graphical representations 
depicts a composite opinion value for a panel of participants 
providing opinion data for the event. 

A method of claim 37, wherein said first of at least two of said 
graphical representations is displayed in a first color and said 
second of at least two of said graphical representations is displayed 
in a second color, said second color being readily distinguishable 
from said first color. 

A method of claim 37, further comprising the step of displaying in 
relation to the event at least one of a video, audio or text. 

A method of claim 40, further comprising the steps of: 

displaying a time line associated with said graphical 
representations, said time line including a plurality of time marks, 
wherein each time mark is associated with one of said plurality of 
designated opinion values and provides an index into said event; 
and 

in response to said participant operating the input device to 
activate one of said time marks, producing a dynamic or static 
display of said video or said text, said video or said text 
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5 corresponding to said designated opinion value associated with 

said activated time mark. 

42. A method of claim 40, further comprising the steps of: 

displaying a time line associated with said graphical 
representations, said time line including a plurality of time marks, 
10 wherein each time mark is associated with one of said plurality of 

designated opinion values and provides an index into said event; 

displaying one or more speaker icons associated with one 
or more speakers engaging in said event, each speaker icon being 
linked to one of said time marks; and 
15 in response to said participant operating the input device to 

activate one of said speaker icons, producing a dynamic or static 
display of said video or said text, said video or said text 
corresponding to said designated opinion value associated with 
said time mark linked to said activated speaker icon. 

20 43. A method of claim 37, further comprising the step of displaying in 

relation to the event a video and captions from said video. 

44. A method for presenting opinion data on a display coupled to a 
data processing device said opinion data collected from a plurality 
of participants in response to an event, comprising the steps of: 

25 displaying a plurality of graphical representations of 

composite .opinion values as a function of time, each graphical 
representation depicting composite opinion values filtered by a 
specific category, said category selected from a group consisting of 
total panel, age, gender, political affiliation, profession, address 

30 and psychographic patterns; and 

displaying a plurality of tables summarizing composite 
opinion values as a function of time, each table depicting 
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45. 



composite opinion values filtered by a specific category, said 
category selected from a group consisting of overall score, event 
segment, event issues, and event topic. 

A method of claim 44, wherein said step of displaying graphical 
representations further comprises the step of displaying each of 
said plurality of graphical representations in at least one of a 
readily distinguishable color or line style. 



46. A method of claim 44, wherein said step of displaying tables 
further comprises the step of displaying within each of said 
plurality of tables composite opinion values filtered by a specific 
sub-category, said sub-category selected from a group consisting 
of total panel, age, gender, political affiliation, profession, address 
and psychographic patterns. 

47. A method for collecting opinions in real time from one or more 
participants in communication with a data processing unit, 
comprising the steps of: 

sending media, from the data processing unit, to one or more client 
apparatuses configured to receive input from a participant, wherein said media 
comprises an event; 

collecting said input and other data related to said participant, 
wherein said input comprises a value indicating a current opinion from said 
participant regarding said event; 

tabulating said input and said other data to generate one or more 
composite results; 

generating one or more graphical representations for at least one of 
said one or more composite results; and 

sending said one or more composite results or said one or more 
graphical representations to said one or more client apparatuses. 
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5 48. A method of claim 47, wherein said event is a live event streamed 

to said one or more client apparatuses over a network connection. 

49. A method of claim 48, wherein said network connection comprises 
a global-based Internet. 

50. A method claim 47, wherein said media includes a plurality of 
10 events and said collecting step further comprises the step of 

collecting a plurality of values, each of said plurality of values 
indicating a current opinion regarding its respective event. 

51. A method of claim 47, further comprising the step of sending 
15 commands, from the data processing unit, to said one or more 

client apparatuses to synchronize a time stamping unit in 
communication with said one or more client apparatuses. 

52. A method of claim 5 1 , wherein said input further comprises a time 
stamp generated from said time stamping unit. 

20 

53. A method of claim 52, wherein said time stamp comprises frame 
codes. 

54. A method of claim 52, wherein said time stamp comprises time 
25 codes. 

55. A method of claim 47, wherein said collecting step includes the 
step of collecting said value each time said one or more client 
apparatuses receives said value from said participant. 

30 
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A method of claim 47, wherein said collecting step includes the 
step of collecting said value at a periodically scheduled rate. 

A method of claim 47, wherein said tabulating step further 
comprises the step of filtering said input and said other data to 
select data that meets at least one specified criterion, thereby 
generating said one or more composite results from said selected 
data. 

A method of claim 57, wherein said other data includes 
demographic data and said at least one specified criterion is related 
to said demographic data. 



59. A method of claim 47, further comprising the step of storing said 
input and said other data according to said participant. 

60. A method of claim 47, wherein said tabulating step further 
comprises the steps of: 

filtering said input and said other data to select data that meets more 
than one specified criterion; and 

generating said one or more composite results for each of said 
specified criteria, thereby creating a plurality of composite results and a plurality 
of graphical representations, thereby sending said plurality of composite results or 
plurality of graphical representations to said one or more client apparatuses. 

61. A method of claim 47, further comprising the step of sending 
instructions to generate a dial control on a display GUI for said one 
or more client apparatuses, wherein said dial control displays a 
graphical metaphor of said value or said one or more composite 
results. 
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62. A method of claim 47, wherein said event is a delayed broadcast of 
a live event streamed to said one or more client apparatuses over 
a network connection. 

63. A method of claim 47, wherein said event is routed on-demand 
from storage media to said one or more client apparatuses over a 
network connection. 

64. A method for collecting opinions in real time from one or more 
participants in communication with a data processing unit, said 
participants generating said opinions in response to independent 
media streams transmitted from one or more stand-alone 
apparatuses, comprising the steps of: 

receiving, at the data processing unit, from one or more client 
apparatuses configured to receive input from a participant, signals indicating said 
participant is receiving the independent media streams from one or more 
stand-alone apparatuses, wherein the independent media streams comprises an 
event; 

collecting said input and other data related to said participant, 
wherein said input comprises a value indicating a current opinion from said 
participant regarding said event; 

tabulating said input and said other data to generate one or more 
composite results; 

generating one or more graphical representations for at least one of 
said one or more composite results; and 

sending said one or more composite results or said one or more 
graphical representations to said one or more client apparatuses. 

65. A method of claim 64, further comprising the step of sending 
commands, from the data processing unit, to said one or more 
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client apparatuses to synchronize a time stamping unit in 
communication with said one or more client apparatuses. 

66. A method of claim 65, wherein said input further comprises a time 
stamp generated from said time stamping unit. 

67. A method of claim 66, wherein said time stamp comprises time 
codes. 



68. A system for collecting opinions in real time from one or more 
participants in communication with a data processing unit, 
comprising: 

media management server for sending media to one or more client 
apparatuses configured to receive input from a participant, wherein said media 
comprises an event; 

polling server for collecting said input and other data related to said 
20 participant, wherein said input comprises a value indicating a current opinion from 

said participant regarding said event; 

analyzing unit for tabulating said input and said other data to 
generate one or more composite results; 

display unit for generating one or more graphical representations 
25 for at least one of said one or more composite results; and 

media routing unit for sending said one or more composite results 
or said one or more graphical representations to said one or more client 
apparatuses. 

69. A system of claim 68, wherein said media includes a plurality of 
30 events and said polling server comprises a plurality of polling 

servers configurable to receive values from said plurality of events. 
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A system of claim 69, further comprising a load balancer for 
simultaneously handling data between a plurality of said client 
apparatuses and said plurality of polling servers. 

A system of claim 69, further comprising a time base generator for 
sending commands to said one or more client apparatuses to 
synchronize a time stamping unit in communication with said one 
or more client apparatuses. 

A system of claim 71, wherein said input further comprises a time 
stamp generated from said time stamping unit. 

A system of claim 68, wherein said media management server 
includes a user interface, said user interface comprising a polling 
management unit for altering the content or mode of said event. 

A system of claim 68, wherein said analyzing unit further 
comprises filtering means for selecting, from said input and said 
other data located within said polling server, said data that meets 
at least one specified criterion, whereby said analyzing unit 
generates said one or more composite results from said selected 
data. 

A system of claim 74, wherein one of said media is a plurality of 
events and said polling server further comprises a plurality of 
polling servers configurable to receive values regarding a plurality 
of events. 

A system of claim 74, further comprising means for storing 
demographic data related to said participant, wherein said at least 
one specified criterion is related to said demographic data. 
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5 77. A system of claim 68, further comprising filtering means for 

selecting, from said input and said other data located within said 
polling server, said data that meets more than one specified 
criterion, whereby said analyzing unit generates said one or more 
composite results for each of said specified criteria, thereby 
10 creating aplurality of composite results and a plurality of graphical 

representations, and sending said plurality of composite results or 
plurality of graphical representations to said one or more client 
apparatuses. 

15 78 - A system for collecting opinions in real time from one or more 

participants in communication with a data processing unit, 
comprising: 

media management server for sending media to one or more client 
apparatuses configured to receive input from a participant, wherein said media 
20 comprises an event; 

polling server for collecting said input and other data related to said 
participant, wherein said input includes a value indicating a current opinion from 
said participant regarding said event; 

analyzing unit for tabulating said input and said other data to 
25 generate one or more composite results; and 

media routing unit for sending said one or more composite results 
to said one or more client apparatuses. 

79. A computer program product comprising a computer useable 
medium having computer readable program code means embedded 
30 m said medium for causing an application program to execute on 

a computer that collects opinions in real time from one or more 
participants in communication with said computer, said computer 
readable program code means comprising: 



BNSOOCIO. <WO 0153922A2 I > 



